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ABSTRACT 

A study gathered information on the estimated 
population eligible to be served by the federal-state vocational 
rehabilitation (VR) program. It contrasted those accepted and those 
not, described services clients received, and evaluated the program's 
outcomes. The study found that, in national surveys in the 1980s, 
14-18 million people reported work limitations that made them 
potentially eligible for VR, but a much smaller group was actually 
served by the state-federal VR program: in any one year, 5-7 percent 
of those potentially eligible. Those accepted were generally similar 
to those who applied, except that those accepted were much more 
likely to be classified as having a severe disability. Most VR 
clients received only modest services. Less than half received any 
type of education or training services, the total value of purchased 
services averaged only $1,573 per client, and just under half 
received purchased services costing less than $500. States purchased 
more services for clients with physical than with mental 
disabilities, more for clients with severe than with nonsevere 
disabilities, and more for White clients than for Black, Hispanic, or 
American Indian clients. Evaluation of long-term outcomes found that 
rehabilitants 1 gains in employment and earnings faded after about 2 
years. (Appendixes include a list of major disabling conditions of VR 
clients, racial differences on variables, regression analyses for 
long-term outcomes, 19 references, and a glossary.) (YLB) 
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GAO 



United States 

General Accounting Office 

Washington, D.C. 20548 



Program Evaluation and 
Methodology Division 

B-252688.2 

August 27, 1993 

The Honorable Msjor R. Owens 

Chairman, Subcommittee on Select Education 

and Civil Rights 
Committee on Education and Labor 
House of Representatives 

Dear Mr. Chairman: 

At your request, we have gathered information to respond to questions about who is served, and 
with what results, by the federal-state vocational rehabilitation program, directed by the 
Rehabilitation Services Administration in the Department of Education. 

We found that only a small fraction of those potentially eligible are served and that those who 
do take part in the program receive, on the average, only modest services. The long-term results 
are also modest 

We are sending copies of this report to officials in the Department of Education and to others 
who are interested, and we will make copies available to others upon request. If you have any 
questions or would like additional information, please call me at 202-512-2900 or Robert L. York, 
Director of Program Evaluation in Human Services Areas, at 202-512-5885. Other mayor 
contributors to the report are listed in appendix VI. 

Sincerely yours, 




Eleanor Chelimsky 
Assistant Comptroller General 
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Executive Summary 



The state-federal vocational rehabilitation (vr) program directed by the 
Rehabilitation Services Administration (rsa) in the Department of 
Education helps persons with disabilities become employed, more 
independent, and integrated into the community. The Chairman of the 
Subcommittee on Select Education of the House Committee on Education 
and Labor asked gao to estimate the eligible population, contrast those 
accepted and those not, describe the services clients received, and 
evaluate the program's outcomes. Examining rehabilitation outcomes 
using long-term data and comparison-group study designs to overcome 
some of the long-standing gaps in this area was a major part of gao's 
purpcje in undertaking this work. 



RQr»lrc*rminH To be eli $ ble for ^ a P erson must have C 1 ) medic al certification of a 

r>aCKgrOUnu physical or mental disabling condition, and (2) evidence that the condition 

is a substantial impediment to employment. In addition, there must be 
(3) a reasonable expectation that vr services will enhance the person's 
employability. (The third criterion was notably modified in the 
Rehabilitation Amendments of 1992, enacted after gao finished work on 
this review.) gao estimated how many persons may have disabilities 
affecting their eriployability, using data from national surveys, gao 
analyzed the mos' recent (1988) complete client data available from rsa to 
find out more about those accepted and those not, as well as the services 
accepted clients received, gao examined the program's long-term results 
using a computer-matched data base on nearly 900,000 vr applicants 
whose cases were closed in 1980, combined with Social Security 
Administration (ssa) wage records on these individuals from 1972 through 
1988— that is, both before and after their program experience, gao 
estimated the program's effect by comparing the employment and earnings 
of three groups of applicants: those who were rehabilitated, those who 
dropped out, and a third group that received services but were not 
rehabilitated. 



V> If* R ' of Answering the first two questions, gao found that in national surveys in the 

KeSUltS in r>nei 1980 , s a yery large &oup Qf Americans— 14 to 18 million— reported work 



limitations that made them potentially eligible for vocational 
rehabilitation. However, a much smaller group was actually served by the 
state-federal vr program— in any one year, 5 to 7 percent of those 
potentially eligible. Those accepted were generally similar to those who 
applied; gao found no mayor disparities at that stage, except that those 
accepted were much more likely to be classified as having a severe 
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disability; however, no data addressed the question of why some never 
applied. 

Answering the third question, gao found most vr clients received only 
modest services: Less than half received any type of education or training 
services, the total value of purchased services averaged only $1,573 per 
client, and just under half received purchased services costing less than 
$500. The appropriateness of services for each individual is the key 
question; rsa data did not allow gao to reach conclusions, but gao did find 
that states purchased more services for clients with physical than with 
mental disabilities, more for clients with severe than with non-severe 
disabilities, and more for white clients than for black, Hispanic, or 
American Indian clients. 

On the fourth question, gao concluded that evidence on vr results was 
mixed. In contrast to the short-term gains typically reported by the 
program, gao's evaluation of long-term outcomes found that rehabilitants' 
gains in employment and earnings from time of referral to their 
case-closure year of 1980 faded after about 2 years. The fraction working 
shrank steadily. By 1988, the last year examined, 61 to 66 percent of 
rehabilitants (depending on type of disability) had some earnings; 
however, this was either no better than or below the pre-program level 
(depending on type of disability), and only a third had worked 
continuously since 1980. Conversely, rehabilitants did do better than 
dropouts on all measures of work and earnings, even after statistical 
analyses controlled for some pre-program differences between the groups. 
Extensive vr efforts were not uniformly effective, however, as shown by 
the gao finding that the group of clients who received significant services 
but were not rehabilitated (21 to 36 percent of those served, depending on 
type of disability) did no better in later employment and earnings than 
dropouts who never got any services after the initial evaluation. 



Principal Findings 



Who Is Potentially Eligible The 14 to 18 million figure of those potentially eligible represented an 
and Who Accepted? upper limit of the population in the mid-1980's, since the data allowed 

estimation of those eligible on only two of the three statutory criteria. 
(Employability cannot be determined using extant data.) Now, however, 
this figure is a more appropriate estimate for the future since the 
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modification of the employability criterion in the Rehabilitation 
Amendments of 1992. About 65 percent of those served in 1988 had sev ire 
disabilities, which is comparable to the 69 percent of the national 
work-disabled population who have severe disabilities. Applicants, 
however, were much less likely to be older (over age 45), to be female, or 
to have disabilities such as orthopedic impairments or chronic health 
conditions, than were persons in the national work-disabled population. 
These findings raise questions about why some disabled-population 
subgroups may not have sought vr services. 



Beyond the two initial services (diagnosis and evaluation, and counseling), 
about half of vr clients also received some type of skill-enhancing service, 
such as education or training. Smaller percentages of clients received 
other services targeted on difficulties associated with their specific 
disabilities. No data were available to allow gao to evaluate whether the 
disparities in purchased services noted previously were appropriate. 



By rsa's definition, about 60 to 70 percent of clients accepted for services 
were rehabilitated — that is, they completed the planned services and then 
held a job for at least 60 days. Some clients held a job before they were 
referred for vr, of course, but more worked for wages immediately after 
closure than before (from 8 to 18 percentage points more, depending on 
the type of disability). Although this wage-earning group shrank in 
subsequent years, average earnings did rise, and rehabilitants continued to 
do better than dropouts. 

gao's statistical analyses to control as much as possible for prior 
differences showed statistically significant positive effects for 
rehabilitants, when compared with dropouts, with rehabilitants more 
likely to be employed and have higher earnings at the 5-year point across 
all three disability groups. Specifically, rehabilitated clients with physical 
disabilities were 12 percentage points more likely to be employed and 
earned about $2,000 more per year; rehabilitants with emotional 
disabilities were 15 percentage points more likely to be employed and 
earned about $1,600 more; and those with mental retardation were 
19 percentage points more likely to be employed and ^arned about $1,000 
more. 

In contrast, clients who were not rehabilitated had long-term economic 
outcomes very similar to those for clients who dropped out. This raises 
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questions about the program's impact because this group, on average, 
remained in the program for as long as rehabilitated clients and received 
up to two thirds of the vr agency-purchased services received by a fully 
rehabilitated client. 



T? p r»nm m pn H 5* ti nn q g a0 recommen ds that the Commissioner of rsa begin the review, 

ttecummtJIlUdllUIlb authorized in the 1992 amendments to the Rehabilitation Act, of the 

adequacy of existing vr data for various users, with particular emphasis on 
measures of the vr referral process and of the cost, intensity, and 
frequency of services. In addition, gao recommends that rsa determine 
why disparities exist in the cost of purchased services for clients of 
different races. To better evaluate the economic impact of the vr program, 
rsa should continue its commitment to a longitudinal study of the vr 
program, and the Secretary of Education and the Secretary of Health and 
Human Services should negotiate an agreement to produce updated 
computer matches of client and earnings data. Finally, to explore the 
broader issues of who can be served, at what intensity, and with what 
results, gao recommends that the Secretary of Education take steps to 
establish the National Commission on Rehabilitation Services authorized 
by the 1992 amendments. The Commission can review gao's findings and 
other up-to-date information on vr outcomes in order to derive 
recommendations for the future direction of the program, particularly for 
the next reauthorization. 



A a Pn r\T Com *n Pn Responsible officials of the Department of Education provided oral 

Agency L/ ^LiLieillb comments on the findings, conclusions, and recommendations in this 

report. Although they raised a number of issues about gao's analysis, in 
general they agreed with gao's recommendations. 
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Chapter 1 



Introduction 



Individuals with disabilities are more likely to be unemployed or living in 
poverty than are Americans without disabilities. One goal of the landmark 
Americans With Disabilities Act of 1990 was to increase the employment 
opportunities for individuals with disabilities by reducing such barriers to 
employment as the inaccessibility of workplaces and the discriminatory 
practices of employers. But many individuals, especially those with severe 
disabilities, are also in need of education, skill training, and other assistive 
services to effectively prepare them to take advantage of work 
opportunities. 

The Rehabilitation Act of 1973, as amended, authorizes the Department of 
Education's Vocational Rehabilitation (vr) program, which provides 
federal funds to help persons with disabilities become employed, more 
independent, and integrated into the community. The federal funds are 
chiefly passed to state vocational rehabilitation agencies that directly 
provide services such as guidance, counseling, and job placement, as well 
as purchase services such as therapy and training from other providers. 1 
The federal share of funding for these services is generally about 
80 percent; the states pay the balance. In fiscal year 1991, $1.6 billion in 
federal funds went to the program, and about 945,000 persons were 
served. 

To be eligible for the program, a person must possess (1) medical 
certification of a physical or mental disabling condition, and (2) evidence 
that the condition is a substantial impediment to employment. In addition, 
there must be (3) a reasonable expectation that vr services will enhance 
the person's employability. 2 The state agencies are also required to focus 
services on individuals with severe disabilities. About 57 percent of the 
applicants in 1990 were accepted for services; of those served, about 
69 percent were classified as severely disabled. 

Traditionally, the vr program has been justified as a good investment, with 
supporters pointing to cost-benefit studies showing high positive ratios of 
earnings gains of former clients to money spent on program services and 
administration. But questions have been raised about whether the program 



•The Rehabilitation Services Administration (RSA) in the Department of Education is responsible for 
the overall administration of the program. There are 83 state agencies in the 50 states, the District of 
Columbia, and U.S. territories and protectorates. In some states, one agency serves all persons with 
disabilities, while in 26 states there are two agencies, with one serving only blind clients. 

*This third criterion was modified in the 1992 reauthorization of the VR program (Rehabilitation Act 
Amendments of 1992, P.L. 102-569). Under the revised act, it is presumed that any applicant's 
employability can be enhanced, unless the state agency can demonstrate otherwise with clear and 
convincing evidence. All the data used in this report are from years before this change. 

12 
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is able to serve all those who are eligible and desire services, whether the 
services provided are sufficient in scope and suitably targeted to meet the 
needs of a diverse clientele, and whether the program's effects persist over 
the long term. 

Effective vocational rehabilitation programs are important for a number of 
reasons. First, a productive and humane society is enhanced by the useful 
employment of a? many of its adult members as possible. Second, 
statistics suggest that the population of Americans with work disabilities 
may be increasing. Some scholars have argued that recent reductions in 
the risk of death from accidents and illnesses are associated with an 
increasing risk of disability. And third, technological developments such as 
the availability of assistive devices and new behavioral training techniques 
have made it possible for individuals who were previously regarded as 
unemployable to enter the workplace. 

Effective vocational rehabilitation programs are thus both more necessary 
and more feasible than in the past. Those responsible for decisions about 
the state-federal vr program now more than ever need information about 
how those with disabilities are being served and what works. 

One uncertainty in designing the program concerns how those with the 
most severe disabilities are treated. On one hand, the law requires the vr 
program to give them priority. Yet critics have argued that the state 
agencies operating the vr program may do the opposite by employing a 
strategy called "creaming" — that is, most frequently accepting for services 
those applicants with greater amounts of work experience or education, or 
with less severe disabilities. Such decisions could be made as a result of 
pressure to achieve the largest numbers of rapid rehabilitations at the 
lowest cost. 

Another persistent issue is the extent of the services provided and the 
accuracy with which services are matched to needs. Once a client is 
accepted, a vr agency provides individualized rehabilitation services that 
follow a written plan drawn up by a rehabilitation counselor in 
consultation with the client. Clients may seek assistance for a number of 
reasons, and the service regimens that are offered vary in both length and 
type. For most clients, a primary reason for seeking rehabilitation services 
is getting or keeping a job in the competitive labor market, doing work 
suited to their abilities. For some, all that may be required is a little 
counseling and guidance, as well as help in finding job opportunities to 
pursue. But for others, employment may depend on the agency's spending 

13 
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money to purchase an assistive device, such as a hearing aid, a wheelchair, 
or a specially equipped van. And for still others, success in a job may 
require an extensive period of higher education, training in social skills, or 
therapy for a problem relating to mental illness or substance abuse. Critics 
have argued that the vr agencies may prefer to give a larger number of 
clients the less expensive in-house services, such as diagnosis, counseling, 
and guidance, rather than (potentially) serving fewer but concentrating 
resources on the more expensive and prolonged services that could give 
clients the skills and technology necessary for long-term success in the 
labor market. 

In addition to uncertainties about who is eligible, who accepted, and how 
they are served, there has been a more fundamental uncertainty 
concerning the long-term vocational outcomes for clients after they leave 
the program, and how former clients compare with persons who are not 
served. A central objective of this study was to analyze the work and 
earnings history of both program participants and nonparticipants for 8 
years after leaving the program, using a previously unexamined data 
source. 



Objectives, Scope, 
and Methodology 



The Chairman of the House Subcommittee on Select Education asked us 
to provide information on several aspects of both the population with 
disabilities and the state-federal vr program, to help in the reauthorization 
of the vr program in the last congressional session. 3 Specifically, we were 
asked to answer the following questions: 

1. How many people with disabilities are potentially eligible for 
rehabilitation services, and what is the nature of this population? 



2. What are the characteristics of those people who receive services? 



3. What types of services are received? 



4. What results are achieved through the delivery of vocational 
rehabilitation services? 

In further defining issues for i*udy, we reviewed literature on vr, 
discussed the program history and the general study questions with 



^his subcommittee was renamed the Subcommittee on Select Education and Civil Rights in the 103rd 
Congress (1993-94). We gave preliminary results of our work in testimony before the Subcommittee as 
it considered the reauthorization. See Vocational Rehabilitation i Program: Client Characteristics, 
Services Received, and Employment Outcomes, GAO/T-PEMD-92-3 (November 12, 1991). 
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congressional staff, and reviewed testimony by witnesses who discussed 
the vr program in detail. From these sources, we gained a better 
understanding of the views of critics and supporters of the program, which 
helped us define questions that we could address with data. 



The Potentially Eligible We used analyses and tables from two published reports to answer the 

Population ^ rst question about the size and nature of the population of people with 

disabilities who potentially were eligible for vr services. 4 One report used 
data from a special supplement on disability done as part of the 1984 
Survey of Income and Program Participation (sipp). 5 The second report 
used aggregated data from three years (1983-85) of the National Health 
Interview Survey (nhis). 6 Both sipp and nhis are surveys of representative 
household samples of the nation's noninstitutionalized civilian population. 
sipp is administered by the Census Bureau. 7 nhis is administered by the 
National Center for Health Statistics (nchs). 8 

We examined estimates derived from these two national surveys from the 
1980's for two reasons. First, of several recent national surveys with some 
information on disability, only nhis and sipp have information on the types 
of disabling conditions prevalent in the population. Second, they contain 
sufficient information and large enough sample sizes to describe the 
population of persons with disabilities in terms of other demographic 



*The March supplement to the Current Population Survey (CPS) has since 1981 contained "questions 
that provide information on work disability status. CPS estimates of the prevalence of work disability 
are lower (8.6 percent of the working age population in 1988, for example) than those from the sources 
we selected and report on in chapter 2. The Bureau of the Census cautions, however, that the 
difference is a result of technical differences in survey methods and that "CPS data are not the best 
source for prevalence estimates." J. Bennefield and J. McNeil, Labor Force Status and Other 
Characteristics of Persons With a Work Disability: 1981-1988 , Current Population Reports, Special 
Studies, Series P-23, No. 160 (Washington, D.C: U.S. Department of Commerce, Bureau of the Census, 
1989). 

5 Assistant Secretary for Planning and Evaluation, Department of Health and Human Services, Task 1: 
Population Profiles of Disability , report prepared by Mathematica Policy Research, Inc. (Washington, 
D.C: October 1989). 

*M.P. LaPlante, Data on Disability from the National Health Interview Survey, 1983-1985 , an Info Use 
Report (Washington, D.C: National Institute on Disability and Rehabilitation Research, 1988). 

7 SIPP is a longitudinal survey. The same households are interviewed every 4 months for 2-1/2 years. 
Core questions are asked in every interview, with questions on specific areas of interest in any one 
wave. The supplement on disability was the third of four waves and was administered between May 
and August 1984. The 1984 panel contains information on persons residing in approximately 20,000 
dwelling units. Each person 15 years of age and older in the household was interviewed individually. 

*The NHIS is a multistage probability design permitting continuous sampling of the population. 
Samples are drawn weekly; each is representative of the target population and is additive with other 
weekly samples. The report we used was based on interviews with 105,620 people in 1983, 105,290 in 
1984, and 91,531 in 1985. 
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characteristics. Thus, information from the surveys allowed us to answer 
the question about the nature of the potentially eligible population, in 
addition to its size. 

nhis and sipp contain items developed for the purpose of monitoring the 
health and economic well-being of the nation's citizens, and the items on 
disability and employment limitations were not specifically designed to 
measure whether respondents would meet the criteria for eligibility useri 
by vr program personnel. However, the reports we examined provide the 
best estimates of the number of persons who potentially meet two of the 
three criteria for eligibility for vr services — that is, persons (1) who have a 
disabling condition, and (2) whose ability to work is substantially limited 
by this condition. 

In both sipp and nhis, respondents were asked whether an impairment or a 
health problem limited their ability to work, or kept them from working 
altogether. Those who reported being limited in working (or prevented 
altogether) were then asked the names of the conditions that caused their 
limitation, and what condition was the main cause. The sipp respondents 
were shown a flashcard with a list of conditions; the nhis respondents 
simply volunteered the condition. In responding to sipp, a condition that 
was either acute or chronic could be named as a cause of work limitation. 
In the nhis data, however, only chronic conditions were recorded. A 
condition was classified as chronic if it had be^n noticed 3 months or 
more before the date of the interview or was on the nchs list of conditions 
that were defined as chronic regardless of time of onset. The exact item 
wordings from the two surveys are presented in table 1.1. 9 



!6 
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t^he questions were posed only to adults between the ages of 16 and 72 in SIPP, and 18 and 69 in 
NHIS. 
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Table 1 Items Measuring Limitations ^BI^^^^Hi^HH^^H^^^HH^^HB^H 

in the Ability to Work, and the Major Survey 

Condition Causing the Limitation | tem siPP NHIS 

Work lim itation 

Question 1 Does your health or Does any impairment or 

condition limit the kind or health problem keep you 

amount of work you can do? from working at a job or 

business? 

Question 2 Does your health or Are you limited in the kind 

condition prevent you from or amount of work you can 

working at a job or do because of any 

business? impairment or health 



problem? 



Cause of work limitation 


Question 1 


(SHOW FLASHCARD) 
What health condition is the 
main reason for your work 
limitation? 


What condition causes this? 


Question 2 


a 


Besides (condition) is there 
any other condition that 
causes this limitation? 


Question 3 


a 


Which of these conditions 
would you say is the MAIN 
cause of this limitation? 



a Follow-up questions not asked in SIPP. 



Sources: Bureau of the Census. Survey of Income and Program Participation (SIPP), 1984 Panel; 
and National Center for Health Statistics (NCHS), National Health Interview Survey (NHIS), 1985. 



The reports we examined presented tabulations of the percentage of the 
population that was limited in or prevented from working, for the entire 
working age population (aged 18 to 64), as well as breakdowns of the 
former population by various demographic characteristics. In addition, 
individuals were categorized according to the msjor condition causing the 
limitation. We relied on the coding of conditions that had been done by the 
authors since we did not conduct our own analyses of the surveys. 



Client Characteristics To answer the second question about the characteristics of those served 

by the vr program, we conducted our own analyses of a m^jor set of data 
called the Case Service Reports. The Rehabilitation Services 
Administration (RSA) routinely collects information from the state agencies 
at the end of each fiscal year on the characteristics of each client whose 
case was closed that year in each state's program, as well as on the general 
types of services that each client received and his or her employment 
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status in both the week of application and the week of case closure. At any 
particular time, rsa may be waiting for original or corrected data from one 
or more states for one or more years; thus, at the time we began our study, 
the most recent full year for which largely complete data were available 
was fiscal year 1988. 

The set of vr clients whose cases were closed during a single fiscal year 
includes two distinct groups. The first consists of those who applied to the 
program but were not accepted for services for one of several possible 
reasons. 10 The second and larger group is made up of persons accepted 
into the program and includes those who during the fiscal year (1) were 
rehabilitated (as defined later in this chapter), (2) dropped out of the 
program before a written plan for rehabilitation had been developed or 
before services had been initiated, or (3) were not rehabilitated after 
receiving at least one (and perhaps several) of the services agreed upon in 
the rehabilitation plan. 11 

We analyzed these rsa records to describe those accepted into the vr 
program (in terms of age, gender, ethnicity, education, type of disabling 
condition, and severity of disability) and to make comparisons with 
persons who applied but were not accepted. Of the 605,872 cases closed in 
1988, we found t^at 58 percent were accepted by the state agencies and 42 
percent applied but were not accepted. Where possible, we also compared 
the characteristics of vr clients with the characteristics of those 
potentially eligible that we derived from sipp and nhis data. 



Services Received by VR To answer the third question about vr services, we used the same Case 
Clients Service Reports for 1988. The state agency is required to list, for each 

client, whether or not a service has been received, using a checklist of 13 
general categories of services. Using this information, we present the 
percentage of clients who received each category of service, as well as the 
average number of service categories they received. 



I0 Some applicants are not accepted because they fail to meet the criteria for eligibility. Other 
applicants' cases are closed before acceptance because they refuse services, move to another state or 
cannot be located, fail to cooperate with agency personnel, become institutionalized, or die. An 
additional subset of applicants may not be accepted after being placed into "extended evaluation." 
Applicants are placed into this status for up to 18 months when agency staff cannot readily certify 
eligibility for VR services and must gather further information before a decision can be made. 

"The Case Service Reports cover clients whose cases were closed any time during the fiscal year. The 
year of application will vary across individuals and may have been much earlier (because of variations 
in the length of time required to determine eligibility and to complete the program). For example, two 
clients may be rehabilitated in 1988, but one may have applied for services in 1983 and the other in 
1987. 
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The agency also reports the total dollar amount spent while a client is in 
the program to purchase services from other providers. This figure is not 
the total dollar cost of all services a client receives, since it excludes those 
provided by the vr agency (such as counseling) or those provided by 
others but not purchased by the agency (such as community college 
tuition covered by student aid, or medical expenses covered by health 
insurance). We present the average cost of purchased services and 
determine whether the overall cost varies by type and severity of 
disability, and by various demographic characteristics. We were 
particularly interested in determining if the amount spent on purchased 
services varied according to whether clients were severely disabled or 
from traditionally underserved or disadvantaged groups, such as women, 
blacks, Hispanics, American Indians, and Asian Americans. 



Rehabilitation In answering the fourth and final evaluation question about the program's 

results, we limited our examination of program results to economic 
outcomes— that is, clients' employment and earnings in the years 
following their participation. The Rehabilitation Act does allow state 
agencies to provide services to clients who are not presently able to 
achieve full-time employment in the competitive labor market. Certain 
clients may be placed in some type of unpaid employment; others may 
achieve greater independence even if they are unable to find or maintain 
employment. Nevertheless, paid work in the competitive labor market is 
still a primary objective for most clients, and a job that pays good wages is 
a central means of their achieving independence and emotional well-being. 

The state vr agencies only collect information and report to rsa on their 
clients' earnings and employment during the week before referral. 12 The 
agencies also report on rehabilitated clients' earnings and employment in 
the week of closure. 13 Analyses of these data usually show substantial 
economic gains between application and case closure for clients who are 
successfully rehabilitated. 14 However, conclusions based on these data 
suffer from two major limitations: lack of any comparison group and short 



l2 Until the mid-1980's, state agencies reported information on earnings for the week before referral. 
Since then, state agencies have reported information for the week before application. 

13 Clients are considered rehabilitated, and their cases are closed, when they have received the services 
listed in the individualized written rehabilitation plan and been suitably employed for a minimum of 60 
days. Clients can also be considered successfully rehabilitated if they achieve other outcomes than 
employment 

"RSA, Comparison of Economic Gains Achieved by Persons with Severe and Non-Severe Disabilities 
Rehabilitated by State Vocational Rehabilitation Agencies in Fiscal Year 1988 (Washington, D.C.: U.S. 
Department of Education, 1990). 
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time-perspective. That is, first, it is difficult to attribute gains to the 
program without comparing the rehabilitated clients with other vr clients 
or comparable individuals with disabilities who are not vr clients. And 
second, stronger conclusions about the effectiveness of the rehabilitation 
would come from longer follow-up of clients than the 60-day period now 
used, rsa's observations have the potential to routinely overestimate the 
impact of VR services on rehabilitants, since most have no earnings in the 
week of application but at least modest earnings during the week of 
closure. 15 The absence of earnings at application is likely to be an 
underestimate of the client's true pre-program potential, while the level of 
earnings at closure has an unknown relationship with long-term 
post-program outcomes. 

In our study, we were able to overcome both these limitations. We were 
able to examine annual earnings of both rehabilitated clients and other 
accepted applicants for a much more extended period — ranging from 
several years before referral to 8 years after their cases were closed. We 
did so by analyzing a special computer-matched file, known in rsa and the 
Social Security Administration (ssa) — the two agencies that created it — as 
the "data link." This file contained information from the vr program on the 
characteristics of all applicants whose cases were closed in 1980 that had 
been combined with information from ssa on the annual earnings of these 
same clients for the period between 1972 and 1988. 

The data link had been created by ssa in the late 1980's in cooperation with 
rsa. Using social security numbers, ssa computer-matched rsa client 
records with ssa's Summary Earnings Record (ser), which contains 
information on annual earnings from wages reported for tax purposes by 
employers or by self-employed individuals. 16 ssa was able to find at least 
some wage record (in the ser data) for 96 percent of the 864,940 vr clients 



,5 In 1988, for example, RSA reported 80 percent of rehabilitants had no earnings at the time of 
application, while only 9 percent had no earnings at closure. 

Ifi The files matched have a number of limitations. First, RSA did not include, for each client, the full set 
of data available (notably, omitting information on services received). Second, there are limits on the 
SSA side as well. For example, SER lists most, but not all, \ *ge income (for example, wages from 
certain types of employers, such as government, are not included). And earnings above the maximum 
taxable base are also not included, although this truncation probably affects few of the persons with 
disabilities we were studying. Other kinds of income (such as disability benefits) are recorded in a 
separate SSA file, the Master Beneficiary Record (MBR). The RSA files were matched with the MBR 
data This income would be pertinent to an evaluation of the overall economic situation of persons 
with disabilities. However, as we were interested chiefly in employment and wages, we did not 
examine this portion of the data base in our review. 

20 
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whose cases were closed in 1980. 17 Neither ssa nor rsa tried to check the 
accuracy of the matched data, such as by cross-checking it with other data 
sources. Such a task was also beyond the scope of our study. 

Because they contain personally-identifiable tax information, access to ser 
files is severely restricted by the Internal Revenue code. However, under 
section 6103(f)(4) of the code, gao can be granted access to tax 
information when it is designated as an agent of a congressional 
committee dealing with tax matters. In this case, we analyzed the data 
contained in the rsa-ssa data link as agents of the Joint Committee on 
Taxation. 

Methodology for Examining The first part of our analysis of the results of the vr program is descriptive. 

Program Impact We present tabulations of the proportion of clients who have any ssa 

record of earnings from employment in the years prior to program referral, 
as well as the 8 years after their cases were closed (1981-88). We also 
present information on the continuity of post-program employment for 
former clients, and use as our measure of continuity the number of 
consecutive years clients had any ssa record of earnings from employment 
after their cases were closed. Finally, we present information on the 
average annual earnings in pre-program and post-program years. We 
present these measures of employment and earnings separately for clients 
with three different kinds of primary dirabling conditions: (1) physical 
disabilities (including orthopedic impairments, amputations, visual 
impairments, hearing impairments, and chronic illnesses such as cardiac 
or circulatory conditions, respiratory conditions, neurological conditions, 
and the like); (2) emotional disabilities (including drug abuse, alcohol 
abuse, or mental illness); and (3) mental retardation. 18 We group clients in 
this way because of the very different barriers to employment that might 
be faced by individuals with these types of disabilities, and the fact that 
such differences would be masked if we only presented information on the 
entire sample. 

The second component in our analysis involved the use of statistical 
procedures to determine whether participation in the vr program was 
associated with better long-term economic results. We compared the 



,7 A valid match was defined as linking a VR client's social security number with even a single year's 
entry in SER for the period between 1972 and 1988. Thus, for some clients for whom a valid match was 
found, there might exist complete earnings data for the entire period we studied, while for others there 
might be some years for which no annual earnings were recorded. 

1R Because of limited computer capabilities, we took random samples from the data set for each of the 
three client groups. The resulting numbers of cases were (1) 41,865, or 8 percent of the clients with 
physical disabilities; (2) 39,823, or 1 7 percent of the clients with emotional disabilities; and (3) 40,558, 
or 50 percent of the clients with mental retardation. 
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outcomes of subsets of clients who participated in the program with the 
outcomes of a group of clients that did not participate. As our outcome 
variables, we chose to determine (1) whether clients (whose cases were 
closed in 1980) had any income from employment in 1985, and if so, 
(2) what ) evel of earnings was achieved. Examining outcomes 5 years after 
program participation enabled us to look at the long-term impact of the 
program and avoid the problems associated with examining an earlier year 
such as 1982 or 1983, during which many persons with and without 
disabilities were out of work due to the recession. 

Analyses of this sort are sensitive to the choice of groups being compared 
and the variables that are included in the statistical analysis. When 
evaluating program impact using data from treatment and comparison 
groups that are not formed through random assignment, impact estimates 
may well be affected by pre-program differences between the groups. The 
problem of selection bias occurs when pre-program differences are 
correlated with both the decision to accept program applicants and the 
outcomes from program participation. In the case of vr, applicants who 
are yoianger and more educated, for example, may have more motivation 
to seek employment and may also be perceived as more motivated by 
program personnel. Thus, they may not only be more likely to be accepted 
into the vr program (because the counselor sees the client as "employable" 
with the provision of services), but may also be more likely to be 
successful in the labor market with or without vr services. The causes of 
their greater success in employment, compared with applicants not 
accepted, would therefore include both preexisting differences and 
participation in the program. 

In an effort to overcome the component of selection bias due to decisions 
made by program personnel, we chose to examine only those clients who 
were accepted for vr services. Within the group that was accepted, we 
identified several subgroups that had different vr 
experiences — naturally-occurring variations that are the practical 
alternative to genuine experimental and control groups. Thus, we 
compared rehabilitated clients with clients who were accepted but 
dropped out. (Dropouts may have left the program before a rehabilitation 
plan was developed, or later but still before receiving any services.) This 
comparison showed whether participants judged successful by rsa 
actually did better, in economic terms, than eligible clients who did not 
participate. We also compared clients who received some services but 
were not classified formally as rehabilitated, with the dropouts. This 
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comparison showed whether those with some participation in the program 
(although termed non-rehabilitated) did any better than those who quit. 19 

These types of comparisons do not rule out the possibility of bias due to 
differences between client groups unrelated to decisions made by program 
personnel We cannot make definitive estimates of the program's effects 
because we cannot be sure that people in the three comparison groups 
(rehabilitants, partial participants, and dropouts) were alike in other ways 
when they were referred to the program. However, we used statistical 
analysis methods (more specifically, two types of regression analysis) to 
take into account as many of the preexisting differences between the 
groups as we could, using measures that were available in the data set. 
The information on clients available in the rsa-ssa file allowed us to 
include in our statistical analysis some other variables that were 
associated with differences in earnings and employment, including 
severity of disability; demographic characteristics such as age, gender, 
education, race, and region of the country; and economic conditions in the 
state where clients were served at the time of program referral and 
closure. 20 Since the three groups of clients might well have different 
patterns of pre-program employment, we also included measures of 
employment and earnings during the year of referral and the year prior to 
the year of referral. 21 Our results are presented in chapter 5, and details of 
the regression analysis are contained in appendix IV. 

In addition to differing in ways we documented from the rsa-ssa data file 
and included in our statistical analyses, the groups could differ on other 
factors that are not measured in the data set. For example, rehabilitants 
may differ from program dropouts in motivation, family support, or the 
availability of alternatives to vr — factors that may be associated with 



ld This category of partial participants comprises persons who were accepted and for whom a 
rehabilitation plan was developed. But in contrast to the group labelled dropouts that never started 
their planned activity, this group began— but did not finish. This could have happened for a variety of 
reasons, including moving away, losing interest, getting services elsewhere, entering an institution, or 
death. Dropouts do get time and attention from VR counselors as they go through initial diagnostic 
procedures leading towards a rehabilitation plan. That may be beneficial even in the absence of any 
further services. Nevertheless, we judged the partial participants worthy of separate analysis since 
they receive substantial purchased services as well (as discussed further in chapter 5), 

^We added these variables to the data file, using state-level information from published sources. It 
would be better to include even more local information, such as county-level economic data, to help 
control even more precisely for the labor market situation of each client. (That is, a client in Los 
Angeles faces not a Califurnia-wide set of economic conditions, but rather a more localized situation.) 
We could not do that analysis, however, since the RSA client information in the RSA-SSA data link did 
not include county of residence or of VR service. 

21 Again, stronger analysis of pre-program differences would depend on access to a richer data set With 
information on the onset of each client's disability, it would be possible to have an even better baseline 
picture of employment and wages. However, this information is not available in the RSA-SSA data link. 

23 
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long-term success in the labor market independent of participation in the 
program. Exploring the possible effect of these unobserved differences 
(that is, controlling statistically for their contribution to vr client 
outcomes) was beyond the scope of our analysis. Our conclusions must be 
considered suggestive, and we recommend further steps to clarify the 
picture of program outcomes, including the use of better data and stronger 
study designs. 

We performed our work in accordance with generally accepted 
government auditing standards. We obtained comments from a number of 
experts in the disability field at different points in our work — such as 
when designing the study, planning details of the outcome analysis, and 
drafting our report — and their names are listed in appendix V. 



Strengths and 
Limitations of the 
Study 



The chief strength of our approach to describing the size and nature of the 
eligible population lies in our use of findings from two different surveys of 
the national population. The chief limitations arise from the fact that the 
surveys were collected for more general purposes than ours. Thus, they 
may not have included in their samples all the types of individuals 
potentially eligible for vr (for example, those in institutions or those under 
18 years of age). Further, the surveys relied on individuals' own reports of 
disability and limitations in working; these may correspond only partially 
with medical assessments of disability and with a vr counselor's judgment 
about the extent to which an applicant's disability is a substantial 
impediment to employment. 

The chief strength of our description of the characteristics of vr clients 
and the services they receive is that we divided the overall client group for 
the purpose of analysis, in recognition of the enormous heterogeneity of 
the vr program. We are limited in this regard, however, by the data states 
are required to provide on each client for the rsa Case Service Reports. It 
would have been very useful to have in each client's record more 
information in the following areas: (1) pre-VR work experience and the 
history of an individual's disability, as well as any experiences with other 
service programs before entering vr; (2) within the program period, details 
on the quality, duration, and intensity of the specific services received; and 
(3) the costs to the state agency of providing services, such as counseling 
and referral, as well as the value of services received as benefits from 
other sources. The key issue is the appropriateness of services to the 
client's situation, and we had no independent information that would 
allow us to perform that evaluation. 

24 
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Our examination of the results of the vr program is the central 
contribution of our report. Ours is the first study to provide national data 
on the long-term economic outcomes for clients who participated in the vr 
program following the passage of the Rehabilitation Act of 1973. There are 
several strengths to our approach. By using the combined rsa-ssa client 
and wage data, we could overcome two of the long-standing shortcomings 
of evaluation in the field: We could study clients over a longer term, and 
we could compare different client groups. In addition although not all 
kinds of income are included, the ssa data on earnings, which are based on 
employer reports, may be more accurate than data reported by clients 
themselves. Further, the very large size of the 1980 case-closure group in 
the rsa-ssa data link allowed us to look at long-term outcomes for those 
with different kinds of disabilities, as well as to include a number of 
variables of interest in our statistical analyses. 

There are, however, a number of limitations in our design. First, as noted 
previously, there are limits, imposed both by the data and by our 
resources, on our search for the size of the program's effects. Better data 
would permit better statistical controls for pre-program differences, but 
more complex statistical models than we could explore might possibly be 
helpful in this regard even with the present data. Second, we can do little 
to explain any outcomes — that is, to say why clients work as much as they 
do or earn the wages they do, particularly whether these are related to 
details of the vr services they received. Service data, though available in 
rsa's Case Service Reports (with many limitations already noted in 
discussing our chapter 3 analyses), were not included in the rsa-ssa data 
link. And no other data are available about clients' work histories that 
could help explain the employment and earnings figures — for example, 
what jobs they held or whether they worked full- or part-time — after 
program closure. 



A 0PT1 p V rnmmPTlt^ We * scusse< * our preliminary findings and conclusions with responsible 

Agency OUiLUUeiLlb officials of the Department of Education while we were preparing our 1991 

testimony, and we incorporated a number of their comments and concerns 
in our ongoing analysis. These officials also provided oral comments on 
our final findings, conclusions, and recommendation*. These comments 
are presented, with gao's response, in appendix L 



Or an n 1 751 ti nn nf th P Chapter 2 uses data from published reports to answer the first evaluation 

VjrgdlUZdUUIl Ul Hie question about persons with disabilities who might be eligible for vr. 

Report 
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Chapters 3 and 4 present our analysis of vr program data on those served 
in fiscal year 1988. Chapter 3 answers the second evaluation question 
about who is served and compares those accepted for vr services with 
those who were not in terms of demographic characteristics and type and 
severity of disability. Chapter 4 again uses rsa data on the 1988 client 
group to answer the third question about services received. The final 
chapter answers the fourth evaluation question concerning program 
results, using data on work and wages through 1988 for various groups of 
rehabilitated and other clients whose cases were closed in 1980. 
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The Population Potentially Eligible for 
Vocational Rehabilitation Services 



The first evaluation question asked us to determine the size and nature of 
the population of people with disabilities who are eligible for vocational 
rehabilitation (vr) services. Not all persons with disabilities are eligible for 
services under the Rehabilitation Act, since the law also requires that a 
person's disability present a substantial impediment to employment and 
that the individual can benefit from vr services in torms of employability. 1 
The judgmental nature of these criteria makes it quite difficult to arrive at 
national estimates of those whom vr could be serving. To answer the 
question, we relied on published reports from two national surveys from 
the mid-1980's. Based on the data in these reports, we estimated the 
number of persons meeting the following two (of three) vr criteria: (1) 
presence of a disabling condition that (2) limits the amount and kind of 
work the person can do. 

From our review of these national survey data, we estimated that about 5 
to 7 percent of the working-age population with work-limiting disabilities 
is served in a year by the vr program. We also found that about 69 percent 
of the general work-disabled population is severely disabled, which is 
slightly higher than the percentage of severely disabled clients 
(65 percent) among those served by the vr program in fiscal year 1988. 

In the first section of this chapter, we present estimates of the size of the 
work-disabled population. In the next three sections, we examine the 
demographic characteristics of this population, the type of conditions 
reported as causing work disability, and the severity of the disabling 
conditions. 



HOW MaHV Are ^ e two surve y s we examined — the 1983-85 National Health Interview 

. f Survey (nhis) and the Survey of Income and Program Participation 

Potentially Eligible? (sipp)— showed between 14.3 and 18 million people with self-reported 

health-related work limitations. 2 This represents 10.1 to 12.5 percent of the 
working age population (those aged 18 to 64). Since about 933,000 to 
1 million persons are vr clients in any one year, this means that about 5 to 
7 percent of the work-disabled population are served by the program. 

The 14.3 to 18-million figure is an imprecise estimate of those eligible, 
however, for at least three reasons. First, eligibility is based on medical 



'As noted in chapter 1, the employability criterion was modified substantially in the 1992 
reauthorization of the program. 

differences in the survey estimates can result from, among other things, differences in the questions' 
wording, the order of questions, or the time period queried. 
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evidence and counselors' judgment, which will differ to an unknown 
degree from applicants' own views; and the surveys' self-reports of 
disability capture only the latter. Second, eligibility depends on a third 
judgment, of employability after services are received, which cannot be 
simulated using the survey data. Thus, there is no way to estimate whether 
all applicants should be viewed— as some advocates did for many years 
and which the law does beginning with the 1992 reauthorization— as 
employable. Third, some people outside the 18 to 64 age group are also 
eligible, as are some of those beyond the household population 
represented by the survey data Whatever the number eligible, it is not a 
definitive guide to the potential demand for vr services for an additional 
reason: Not all will want to enter the labor market. (Although there are 
other benefits and outcomes from vr services, they are primarily aimed at 
helping clients achieve jobs in the competitive labor market.) 



??!T^ Table 2.1 shows that those reporting a work disability were older, less 

W HO Are I nOSe educated, and poorer than the general working-age population. The 

Reporting a Work work-disabled population also differed from the working-age population, 

Disability? although less markedly, in race and gender. 
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Table 2.1: Demographic Comparisons 
of Work-Disabled and Working-Age 
Populations 





Work-disabled* 


Working-age b 


Characteristic 


NHIS 


SIPP 


NHIS 


SIPP 


Age 


18-24 


8% 


c 


20% 


c 


25-44 


34 


c 


49 


c 


45-64 


58 


c 


31 


c 


Some education beyond high school 


21 


22% 


37 


38°/ 


Family income below poverty level d 


18 


21 


10 


12 


Gender 


Male 


49 


47 


48 


49 


Female 


51 


53 


52 


51 


Ethnicity 


Black 


14 


14 


11 


11 


Hispanic origin 


5 


5 


6 


6 



Note: Survey sample sizes are given in chapter 1 . 

a Report that a health condition or impairment limits the amount or kind of work they can do. 
b For characteristics other than age, NHIS includes those aged 18 to 69. 
c Data not broken down into these categories. 

d ln NHIS, respondents are categorized as below the poverty level; in SIPP, respondents a r e 
categorized as at or below the poverty level. 

Sources: Assistant Secretary for Planning and Evaluation, Task 1 : Population Profile of Disability , 
report prepared by Mathematica Policy Research, Inc. (Washington, D.C.: U.S. Department of 
Health and Human Services, October 1989); and M. P. LaPlante, Data on Disability from the 
National Health Interview Survey, 1983-1985 (Washington, D.C.: O.S. Department of Education, 
IMS). 



Not surprisingly, the two populations differed considerably in age. For 
example, in the 1980's survey data, 58 percent of those reporting a 
work-limiting disability were aged 45 to 64, while only 31 percent of the 
total working-age population were in this age bracket 

Also, in terms of education and income, persons who reported having 
work limitations differed considerably from the working-age population as 
a whole. In both surveys, about one fifth of the work-disabled group had 
some formal education beyond high school, compared with about two 
fifths of the overall working-age population. In addition, 18 percent of the 
work-disabled group had family incomes that fell below the poverty level, 
compared with 10 percent of the working-age population. 
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Women were about as likely as men to report that a health condition or 
impairment limited their ability to work. Women constituted 53 percent of 
the work-disabled population versus 51 percent of the working-age 
population. 

Black Americans also had a slightly greater representation in the 
work-disabled population than in the working-age population: 14 percent 
versus 11 percent. Persons of Hispanic origin, on the other hand, were 
slightly less represented: 5 percent of the work-disabled population versus 
6 percent of the working-age population. 



WTYi i- A tVi Despite differences in the surveys' methods, they showed considerable 

Wnat Are tlie consistency in the predominant disabling conditions reported as the main 

Work-Limiting cause of work limitation^. Musculoskeletal and cardiovascular conditions 

rnnrHtirm<S? together accounted for 55 to 61 percent of the total, as can be seen in table 

. 2 2 largest category, musculoskeletal impairments (38 to 41 percent 

of all conditions reported), included arthritis, back iryuries and disk 
disorders, spinal cord injuries or deformity, and amputation or absence of 
one or more of the mayor extremities. The next largest category, 
cardiovascular and circulatory conditions (17 to 21 percent of all 
conditions reported), included heart disease, hypertension, and stroke. 
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Table 2.2: Conditions Reported as the BH^H^HIHHI^IHHHHHHBHIHIHBIIHMHHBi^^^^HI^B 

Main Cause of Work Limitation Condition NHIS* S»PP b 

Musculoskeletal 407% 38.1 % 

Cardiovascular/circulatory 20.5 16.9 

Mental illness, mental retardation, substance abuse 5^ 8.3 

Respiratory 7.2 6A_ 

Visual and hearing impairments 5^ 4.2 

Neurological ^3 3.6 

Neoplastic 2^ ^6 

Digestive 27 ^5 

Other conditions d 117 17.3 

Total* 100.0% 99.9% 

a NH!S conditions asked of respondents 18 to 69 years old, 

b SIPP conditions asked of respondents 18 to 64 years old. Percentages are based on all 
respondents who named a specific condition as a cause of their work limitation. About 7 percent 
named no specific condition. 

c Cancers and tumors. 



includes such conditions as diabetes, end-stage renal disease, genito-urinary disorders, 
endocrine conditions, other ill-defined conditions, and all other health conditions. 

e SlPP total does not add to 100 percent due to rounding. 

Sources: Assistant Secretary for Planning and Evaluation, Task 1 : Population Profile of Disability , 
report prepared by Mathematica Policy Research, Inc. (Washington, D.C.: U,S, Department of 
Health and Human Services, October 1989); and M.P. LaPlante, Data on Disability from the 
National Health Interview Survey, 1983-1985 . (Washington, D.C.: U.S. Department ot Education, 
T558£ 



Mental illness, emotional problems, mental retardation, and substance 
abuse together accounted for 6 to 8 percent of all conditions. Respiratory 
conditions, including asthma and emphysema, accounted for 6 to 
7 percent of all conditions. And visual and hearing impairments accounted 
for 4 to 5 percent of all conditions. 



How Severe Are 
These Conditions? 



Both the Rehabilitation Act and the regulations implementing it direct 
state vh agencies to give first priority to serving the severely disabled, so 
we looked for evidence of the size of that particular group. However, 
differences between the two surveys' definitions of the degree of limitation 
caused by a disability and the vh program's administrative definition of 
"severely disabled" made it very difficult to first estimate the total 
population of persons with severe disabilities and then compare it to the 
overall vr population. 
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Estimates derived from sipp were closest to those used by rsa to categorize 
persons as severely disabled. 3 One study we examined reported answers 
to several questions that approximate a measure of severity: Add the 
number of persons who said that their health condition or impairment 
limited the kind or amount of work they could do, and who said that 
(1) they received disability benefits — Social Security Disability Insurance, 
Supplemental Security Income, or Veterans' Administration disability 
benefits; or (2) that they had difficulty with such tasks as personal 
hygiene, getting in and out of bed, dressing and undressing, doing light 
housework, getting around outside the house, or with such functions as 
seeing and hearing, lifting things, walking short distances, or climbing 
stairs without resting. 4 

Based on the sample surveyed by sipp, an estimated 18 million persons 
have some work limitation. Around 69 percent of these (12.4 million) 
could be considered severely disabled based on reported limitations in 
functioning (8.4 million) or receipt of disability benefits (4.0 million). This 
is slightly higher than the 65 percent categorized as severely disabled in 
the vr clientele in 1988. 



p i • The population of persons with work disabilities who are potentially 

UOnClUblOIlb eligible for vr services is large, between 14 and 18 million people, 

comprising over 10 percent of the working age population and including 
persons with a diverse range of disabilities, A majority of these persons 
(about 69 percent) have severe disabilities that limit their capacity to carry 
out activities at home and at work, and a majority are older (that is, over 
the age of 45). Both these factors may be associated with lower 
probabilities of success in the labor market, unless appropriate vocational 
rehabilitation services are provided. 

However, this self-reported population that is potentially eligible for vr 
services might not be the same as the population that would have met the 
vr criteria for eligibility (in use during the period of our review, but now 
changed), which are based on medical evidence, counselor judgment, and 
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3 RSA directs state VR agencies to categorize an applicant as "severely disabled" if he or she ( 1) has a 
major disabling condition such as blindness or deafness, which is automatically included, or other 
disabilities as qualified, such as respiratory disorder with sufficient loss of breathing capacity; (2) is a 
recipient of Social Security Disability Insurance or Supplemental Security Income for reason of 
blindness or disability, at any time during the rehabilitation process; or (3) has documented evidence 
of substantial loss of function in conducting certain specified activities. 

'Assistant Secretary for Planning and Evaluation, Task I: Population Profile of Disability , report 
prepared by Mathematica Policy Research, Inc. (Washington, D.C.: U.S. Department of Health and 
Human Services, October 1989), 
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employability. The exact size of the latter population is difficult to 
estimate. However, with respect to those who are potentially eligible by 
virtue of their self-reported degree of disability, the data show that only 
about 5 to 7 percent of the population with work disabilities is currently 
being served in a year by the state-federal vr program. 
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The second evaluation question asked us to describe who gets vocational 
rehabilitation (vr) services. This question is of great importance to many 
disability advocates, who have charged that vr counselors tend to follow 
decision rules denoted by the shorthand term "creaming" during the 
eligibility review process. That is, according to this analysis, counselors 
accept the easiest cases— those individuals with greater amounts of work 
experience or education, or with less severe disabilities — because of 
pressures to achieve quicker and less costly rehabilitations. More 
sympathetic observers have pointed to the difficulties counselors face in 
predicting the rehabilitation "potential" (the eventual employability) of any 
individual applicant. To address the issue of whether applicants who are 
accepted differ in systematic ways from those who are not, we compared 
the two groups on a variety of demographic and disability-related 
characteristics, using data from the 1988 rsa Case Service Reports. 

In addition, we explored the possibility that applicants differ from the 
general work-disabled population. Observed differences could result from 
differences in motivation, but could also suggest that some persons with 
disabilities who want services and could benefit from them are not even 
reaching the first step of the process. We investigated this possibility by 
comparing the vr client group in the 1988 rsa Case St^vice Reports with 
the general work-disabled population described in the previous chapter. 

We found generally that applicants who were accepted were similar to 
applicants who were not, on many demographic characteristics. There 
were some small differences between the groups in terms of the type of 
disabling conditions that were the main causes of work limitations. And, 
contrary to the creaming argument, applicants who were accepted were 
more likely to have a severe disability or a secondary disabling condition 
than applicants who were not accepted. 1 

We found larger differences between the pool of applicants and the 
general work-disabled population. Some groups that could face greater 
difficulties in the labor market, and thus be considered more "difficult" to 
rehabilitate— including women, older people, and persons with orthopedic 
disabilities and chronic health conditions— were less likely to apply to the 
vr program than their numbers in the general work-disabled population 
would suggest. On the other hand, other groups that also encounter 
difficulties in the labor market— blacks, persons with no formal education 
beyond high school, and persons with sensory impairments and mental or 

■This generalization concerns aggregate national data. In another study, however we found significant 
variation among the states in the extent of persons with severe disabilities in the VR case load. See the 
more detailed discussion in note 4 in this chapter. 
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emotional conditions — were more likely to apply to the program than their 
numbers in the general work-disabled population would suggest. Further 
study of the vr referral process is essential to an understanding of whether 
some persons who could benefit from the vr program are being 
systematically discouraged from applying. 

In the first part of this chapter, we present information on the 
demographic characteristics of applicants (accepted and not accepted) 
and the general work-disabled population. In the second part of the 
chapter, we present information for the same groups on type and severity 
of disability. 



Those accepted and not accepted into the program differ only slightly in 
demographic characteristics (as shown in table 3.1). About 58 percent of 
the applicants were men. Most clients were under age 45, with about a 
quarter aged 18 to 24 and one half in the 25 to 44 range. (The remaining 
25 percent were over 45 or under 18.) Around 20 percent of the applicants 
were black, and 5 percent were of Hispanic origin. Of all applicants, about 
45 percent had less than a high school education. 



Demographic 
Characteristics 
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Table 3.1 : Demographic 
Characteristics of VR Applicants and 
the U.S. Work-Limited Population 





VR applicants* 


Percent in 
U.S.work- 


Demographic characteristic 


Percent 
accepted 


Percent not 
accepted 


limited 
population 5 


Gender 


Male 


58 


59 


48 


Female 


43 


41 


52 


Age 








14-17 


9 


7 


c 


18-24 


25 


24 


8 


25-44 


49 


52 


34 


45-64 


15 


16 


58 


65 and older 


2 


1 


c 


Ethnicity 


Black 


19 


21 


14 


Hispanic 


5 


6 


5 


Years of education 








Up to 1 1 years d 


44 


44 


43 


High school grad 


39 


41 


35 


More than high school 


17 


15 


22 



a Based on GAO tabulations from 1988 RSA Case Service Reports. 

b Entries in this column are averages of the 1984 SIPP and the 1983-85 NHIS percentages (except 
for age. which is derived from the NHIS data only). 

c NHiS data not reported in comparable age category. 

d ln the RSA data, this category includes clients with primary or secondary disability of mental 
retardation, who are categorized as "special education" on the education variable. 



These data provide no evidence that applicants in any of the demographic 
groupings that we examined were accepted or not accepted in 
disproportionate numbers. There were some differences between the 
overall applicant pool and the work-limited population in the United 
States, however. On the one hand, fewer women and fewer persons over 
the age of 44 apply to the program than are represented in the general 
work-disabled population. On the other hand, more blacks and more 
persons without education beyond high school apply to the program than 
are represented in the general work-disabled population. Thus, there is 
mixed evidence that groups that could be considered more "difficult" to 
rehabilitate are also less likely to apply to the vr program than their 
numbers in the general work-disabled population might suggest. 
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Type of Disabling 
Condition 



The state vr agencies serve persons who have a wide range of health 
conditions or impairments that limit their ability to work. Again, we found 
only a few differences between persons accepted and persons not 
accepted into the program with regard to the prevalence of specific 
conditions that were the primary causes of work limitation. In comparison 
with persons not accepted into the program, those who were accepted had 
a greater prevalence of mental retardation (13 versus 8 percent) and 
hearing impairments (7 versus 4 percent), and a lesser prevalence of 
miscellaneous conditions (19 versus 23 percent). These percentages 
appear in table 3.2. 



Table 3.2: Type and Severity of 
Work-Limiting Conditions Among VR 
Applicants and the U.S. Work-Limited 
Population 





VR applicants" 


Percent in U.S. 


Type and severity of disabling 
condition 


Percent 
accepted 


Percent not 
accepted 


work-limited 
population 6 


Sensory (total) 


14 


11 


5 


Visual 


7 


7 


c 


Hearing 


7 


4 


c 


Orthopedic/amputee (total) 


24 


27 


39 


Mental and emotional conditions 
(total) 


43 


41 


7 


Mental illness 


19 


20 


c 


Substance abuse 


11 


13 


c 


Mental retardation 


13 


8 


c 


All other conditions (total) 


19 


23 


49 d 


Severely disabled 


65 


35 


69 e 


Secondary disabling condition 


42 


23 


c 



a Based on GAO tabulations from 1988 RSA Case Service Reports. 

b Entries in this column are averages of the 1984 SIPP and the 1983-85 NHIS percentages. 

c Category not reported in the surveys. 

includes chronic health conditions such as cardiovascular and circulatory conditions, respiratory 
conditions, digestive conditions, neoplasms, neurological conditions, diabetes, and all other 
conditions not elsewhere classified. 



Percentage derived from SIPP only. 



Although precise comparisons between vr clients in 1988 and the U.S. 
work-limited population are impossible because of the different data 
collection and coding procedures employed, there are some differences 
between the two populations in general categories of disabling conditions. 
vr clients are much more likely to have mental, emotional, and cognitive 
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conditions than are individuals in the general U.S. work-lhnited population 
(43 percent versus about 7 percent). 2 We also found a larger percentage of 
sensory conditions (visual and hearing impairments) in the vr client pool 
in 1988 than in the U.S. work-limited population. 

In contrast, vr clients are less likely to have "other" disabling 
conditions — which include such conditions as cardiovascular and 
respiratory diseases, diabetes, and neoplasms, or orthopedic impairments 
(musculoskeletal impairments)— than are those in the U.S. work-limited 
population (24 percent for orthopedic impairments, and 19 percent for 
"other" impairments, compared with 39 and 49 percent, respectively). 
These differences may be attributable to the difference in the average age 
of the two populations. As we have seen, vr clients tend to be younger 
than the U.S. work-limited population. The conditions that are more 
prevalent in the U.S. work-limited population are also those that tend to be 
associated with age. 3 



State vr agencies classified, in total, 65 percent of all those accepted as 
severely disabled, a higher rate than in the group of persons not accepted 
(of whom only about one third were severely disabled). 4 In addition, 
42 percent of those accepted had a secondary disabling condition, 
compared with 23 percent in the group of persons not accepted. 

Although the data sources are not directly comparable, the percentage of 
vr applicants with severe disabilities who were accepted for services in 



20n one hand, it is possible that this observed difference would not hold up under closer study; such 
conditions may be underreported in population surveys. On the other hand, the difference may reflect 
VR referral practices. That is, there may be better referral methods or particular VR program options 
for this group that could explain a genuinely higher representation in the VR caseload. 

3 For example, the prevalence of cardiovascular and circulatory conditions rises dramatically with age. 
In NHIS, 27 percent of those 45 to 69 years old who report a work limitation say that the primary cause 
is a cardiovascular or circulatory condition. Only 8 percent of work-limited persons aged 18 to 44 
attribute their work limitation to these conditions. 

4 ln another evaluation, we found that states varied in the percentage of cases classified as persons 
with severe disabilities, with a range from 29 to 96 percent Despite the requirement in the 
Rehabilitation Act to focus services on this group, some state officials perceived conflicting goals. 
They told us that increasing service to persons with severe disabilities could significantly reduce the 
overall number of clients they could serve. Because in that study we also found RSA guidance was 
unclear and the agency was not adequately checking states' decisions in this regard, we recommended 
both stronger program guidance and increased oversight, which RSA agreed to and has begun to 
implement See Vocational Rehabilitation: Clearer Guidance Could Help Focus Services on Those With 
Severe Disabilities, HRD-S2-12 (November 26, 1991). In the present study, we did not examine data 
below the national level. Considering the importance of the classification, it is of interest whether the 
reported data rest on valid and reliable measures used by states to categorize clients as severely 
disabled, and whether the categorization procedure is applied consistently to accepted and rejected 
clients However, wo had no practical alternative to using the existing data. 
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1988 was slightly smaller than the percentage of severely disabled persons 
in the U.S. work-limited population (65 versus 69 percent). 



Overall, we found little evidence that certain types of applicants were 
disproportionately more likely than other types to be accepted for services 
(based on comparisons between applicants accepted and those not 
accepted). The msyor exception was that individuals with severe 
disabilities were — in the aggregate — more likely to be accepted than were 
individuals with less severe disabilities. (While the program's decisions 
thus generally tend to meet the legal requirement that services be focused 
on those with severe disabilities, we found in another study that states 
vary widely in this regard.) 

We did find several differences between the pool of accepted clients and 
the U.S. work-limited population when we compared our findings from the 
1988 rsa data with information from nhis and sipp. From these data, there 
is no way to tell why certain groups of people were more likely to seek out 
vr services than were others. The fact that certain types of persons choose 
not to apply for services, or are not referred for services, may not be 
evidence of a failing in the vr system itself. These observed disparities may 
be due to variations in individual motivation, or in the availability of jobs 
for certain groups of people. On the other hand, they may instead be due 
to practices of vr agencies or the agencies that are sources of referral to 
the program. These agencies may discourage certain types of people from 
following through with the application process. The findings are 
suggestive of possible problems in the vr referral process, but conclusions 
about the reasons are speculative in the absence of additional data. 



Ppr*nmmpnHstirmQ We recommend that be § in the data rev *ew authorized by the 1992 

tieCOIUIlieilUdllUIlb amendments to the Rehabilitation Act and work as well with the National 

Commission on Rehabilitation Services, if it is established, to develop 
plans that could improve information in the area of disability and 
rehabilitation. Currently, rsa's knowledge of how well it is serving the 
general work-disabled population is inadequate because data limitations 
make precise comparisons between this population and the vr applicant 
pool difficult. Specifically, there are two irayor data. limitations: 
(1) surveys of the general population are too small to allow for state-level 
breakdowns by demographic and disability characteristics (and thus state 
agencies that have no other data cannot rely on such surveys for their own 
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planning purposes), and (2) data on vr clients are limited with regard to 
severity of disability. . 

rsa need not by itself remedy the first of these. However, based on further 
review of the existing data and options for improvement, rsa could take 
action. For example, rsa could encourage new efforts by the agencies in 
charge of monitoring the nation's health and economic well-being, even 
though we could not in our review develop specific plans and check their 
feasibility. Some states may already have methods for describing their 
populations of persons with disabilities. If these methods are sound, they 
could be expanded to other states (with rsa providing technical assistance 
and incentives for data quality and comparability). Establishing a national 
program of periodic special studies is another possibility. 

Remedying the second limitation — by improving rsa's definitions and 
measures in order to strengthen comparisons of data on vr applicants and 
clients with data from other sources on persons with disabilities — is a 
difficult conceptual and technical problem that should also be addressed 
by rsa and the National Commission, rsa could develop measures of the 
severity of disability that are more comparable with those collected by the 
Census Bureau and the National Center for Health Statistics — in 
particular, by paying greater attention to the functional limitations faced 
by vr clients. It is possible, however, that there are better and more 
comparable measures already in place in some states that could be more 
widely adopted. As in the case of population surveys, the scope of our 
review did not permit us to review state-level data to find promising 
practices nor to evaluate the costs and feasibility of new options. Whether 
to augment regular data-gathering and reporting— in this case, adding 
better measures of severity and more comparable classifications of 
disability— to states' existing data routines and reports, or whether to get 
better data in periodic special studies, is again an issue. 

In addition, we recommend that rsa collect additional data on the referral 
process itself in order to determine why certain groups are less likely than 
others to apply for vr services. At this point, few relevant data are reported 
to rsa by the state agencies, other than their recording from a list of 
agencies the source of a referral. (States may collect, but not report, more 
such data.) From the data rsa has, one can learn little about how and why 
certain types of applicants seek rehabilitation services, the alternatives to 
vr that are available to them, how they access the vr system, and their 
treatment by referral sources and vr agencies. 
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If properly done, an evaluation now in progress could help answer some of 
these questions. The 1992 amendments to the Rehabilitation Act 
transferred evaluation authority from the rsa Commissioner to the 
Secretary of Education, as well as required the Secretary to continue a 
longitudinal study of the vr program. We recommend use of a design in 
that evaluation that will trace potential clients from an early point in order 
to shed light on questions like those previously mentioned concerning the 
referral stage. 
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The third evaluation question asked us to describe what types of services 
are received by vocational rehabilitation (vr) clients. The state vr agency 
provides some services, such as guidance, counseling, and job referral; the 
state agency also purchases services from other providers, such as 
diagnosis, psychotherapy, vocational training, and medical restoration; 
and it helps locate certain services that are paid for by other agencies. 
Rehabilitation counselors work with clients to develop an individualized 
service plan that will combine a variety of activities to help clients achieve 
their employment potentials. (Clients can also be considered successfully 
rehabilitated if they achieve outcomes other than employment.) 

But what kind and quantity of services should be provided? Critics of the 
federal-state VR system have argued that vr agencies spend too much time 
and effort on such agency-provided services as guidance and counseling, 
at the expense of providing or purchasing educational and training 
services that could enhance skills necessary for long-term success in the 
labor market. These critics suggest that incentives for counselor 
performance, historical patterns of service provision, and the professional 
training of rehabilitation counselors may contribute to a paternalistic 
relationship between counselor and client, with the result being the 
provision of low-cost service that often may not fit the individual client's 
needs and capabilities. 

Other critics have expressed concerns about whether these same factors 
have resulted in inequitable treatment for some traditionally underserved 
groups. These groups include individuals with more severe disabilities, 
those with mental and emotional disabilities, and minorities and women. 

In this chapter, we address these issues by analyzing data in the 1988 rsa 
Ca^e Service Reports. For all accepted clients whose cases were closed in 
1988 (344,865 individuals), state agencies recorded the general categories 
of services received (whether provided by the vr agency or another 
source), as well as the total cost of all purchased services. As discussed in 
chapter 1, the data have significant limitations and do not fully reveal 
either the kind, quantity, and quality of all services— or their cost. 

In brief, we found that services purchased by the state agency were 
relatively modest in cost (with an overall average of $1,573). Since every 
new client requires initial evaluation, the records not surprisingly show 
that most received diagnosis and evaluation, as well as counseling and 
guidance. About one half of all clients received some type of 
skill-enhancing education and training service. Examining services by type 
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of disability, we found that the average amount spent on purchased 
services for persons with mental and emotional disabilities was less than 
was spent for persons with physical disabilities. Finally, for the most part, 
the average spent on purchased services for clients who were members of 
minority groups was less than that spent for clients who were white. 

In the first part of the chapter, we present our findings on the types of 
services received by vr clients. The number of services received and the 
cost of all purchased services are then tabulated by disability type, severity 
of disability, gender, age, education, and race/ethnicity. 



Types of Services 
Received 



rsa data show that clients commonly received the two initial services that 
would be expected: (1) diagnosis and evaluation, and (2) counseling and 
guidance. A much smaller percentage of the clients received services in 
the other nuyor categories, ranging from 33 percent who were receiving 
restoration services to 8 percent receiving on-the-job training. However, 
47 percent of all clients received at least one of the five categories of 
education and training. These percentages are shown in figure 4.1. A 
description of the specific types of services that make up each of these 
more general service categories appears in the glossary. 
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Figure 4.1: Percent of All VR Clients Receiving Each Category of Service 



100 Percent 




There is some evidence that patterns of service-provision vary according 
to whether a client is or is not severely disabled, and also according to the 
clients major disabling condition. (That is, we found variation in services 
beyond the two most common ones — diagnosis and evaluation, and 
counseling and guidance.) Concerning differences by severity, clients with 
severe disabilities were more likely than those with non-severe disabilities 
to receive services of all kinds, but especially adjustment training. 
Concerning service differences by type of disability, we found that 

• clients with visual impairments received referral and placement services 
least often; 

• restoration services most often went to clients with hearing impairments, 
who conversely least often received transportation services; 
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• a greater percentage of clients with mental retardation received placement 
services, adjustment services, and on-the-job training, and a lesser 
percentage received restoration services and coUege/university training; 
and fuially, 

• a greater percentage of clients with substance abuse problems received 
counseling and guidance, transportation, income maintenance, and other 
types of service. 



The differences in services received, by disability and severity, are shown 
in table 4.1. 



Table 4.1: Percent of Clients Who Received Categories of Service, by Type and Severity of Disability 

Type of major disabling condition 



Severity of 
disability 



Category of 
service 


All VR 
clients 


Visually 
impaired 


Hearing 
impaired 


impairment/ 
amputation 


Mentally 

ill 


Mentally 
retarded 


Substance 
abuse 


Other 
conditions 


Severe 


Non- 
severe 


Diagnosis 
and 

evaluation 


87 


90 


90 


86 


86 


89 


87 


87 


88 


86 


Counseling 
and guidance 


73 


70 


68 


7 


75 


73 


82 


72 


75 


70 


Restoration 


33 


53 


64 


34 


30 


10 


29 


35 


35 


30 


Transportation 


27 


25 


16 


25 


31 


29 


37 


22 


28 


26 


Placement 


22 


15 


24 


20 


23 


30 


22 


21 


23 


20 


Referral 


21 


12 


20 


19 


23 


26 


23 


20 


22 


18 


Income 
maintenance 


20 


18 


14 


19 


23 


18 


28 


17 


20 


19 


Adjustment 
training 


19 


29 


11 


10 


22 


39 


17 


14 


23 


12 


Business or 

vocational 

training 


12 


6 


8 


14 


16 


7 


13 


12 


12 


12 


Miscellaneous 
training 


12 


17 


10 


10 


12 


16 


13 


11 


14 


9 


College or 

university 

training 


11 


9 


12 


15 


11 


2 


10 


11 


10 


" 1 


On-the-job 
training 


8 


7 


5 


5 


7 


15 


7 


7 


9 


6 


Other 
services 


22 


27 


24 


21 


22 


18 


32 


18 


23 


21 



Source: GAO analysis of 1988 RSA Case Service Reports 
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Number of Services 
Received 



The average client received 3 to 4 services out of the 13 services rsa lists in 
the service record, and there were few m^jor differences between groups 
of clients. Two groups stood out from the rest. First, persons of Hispanic 
origin received somewhat more — an average of 4.4 services. (Although as 
we report in table 4.3, the average amount spent on purchased services for 
this group was somewhat lower than the average for all clients.) Second, 
persons over the age of 65 received somewhat fewer — an average of 3.1 
services. These averages are presented in tables 4.2 and 4.3. It should be 
noted that the number of types of services received, however, provides 
indication neither of the number of specific services received nor of their 
intensity or duration. 



Table 4.2: Number and Cost of 
Purchased Services, by Type and 
Severity of Disability 



Type of disability 


Average number 
of service types 


Average cost of 
ail purchased 
services 


Visual impairment 


3.8 


$2,401 


Hearing impairment 


3.6 


1,744 


Orthopedic impairment/ 
amputation 


3.5 


1,920 


Mental illness 


3.8 


1,224 


Mental retardation 


3.7 


1,478 


Substance abuse 


4.0 


975 


Other condition 


3.4 


1,536 


Severity of disability 


Severe 


3.8 


1,798 


Non-severe 


3.4 


1,175 


All clients 


3.7 


1,573 



Source: GAO analysis of 1988 RSA Case Service Reports 
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Table 4.3: Number and Cost of 
Purchased Services, by Client 
Background 



Client background 


Average number 
of service types 


Average cost of 
ail purchased 
services 


Gender 


Male 


3.7 


$1,559 


Female 


3.6 


1,593 


Age 


Under 18 


3.3 


1,630 


18-24 


37 


1,813 


25-44 


3.8 


1,487 


45-64 


3.5 


1,477 


Over 65 


3.1 


865 


Race/Ethnicity 


White 


3.6 


1,642 


Black 


3.8 


1,349 


American Indian 


3.9 


1,263 


Asian American/ Pacific islander 


3.4 


1,622 


White Hispanic 


4.4 


1,472 


Years of education 


Special education 


37 


1,508 


Fewer than 9 


3.4 


1,580 


9-11 


3.6 


1,575 


12 


3.6 


1,562 


13-15 


3.8 


1,626 


16 or more 


3.7 


1,747 


All clients 


37 


1,573 



Source: GAO analysis of 1988 RSA Case Service Reports 



Cost of Purchased 
Services 



On average, the state vr agencies spent $1,573 in purchased services per 
client while their cases were active. However, the program spent less than 
$500 on purchased services for about half (47 percent) of all clients. There 
was a wide range of spending on services for the other clients: $500 to 
$999 for 16 percent, $1,000 to $2,999 for 23 percent, and $3,000 or more on 
15 percent of all clients. 

As might be expected, more money (about 14/2 times more) was spent on 
purchased services for the average client with a severe disability than for 
the average client with a non-severe disability. As table 4.2 shows, more 
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was spent on purchased services for the average client with a physical 
disability (visual, hearing, and orthopedic impairments, and persons with 
"other* conditions or impairments) than for the average client with a 
mental disability (mental illness, mental retardation, and substance 
abuse). The four physical disability categories comprised the top four 
categories in terms of average amounts spent on purchased services. 

In table 4.3, we present the differences in average service costs by client 
demographic variables. We found that the largest differences between 
groups in the average costs of purchased services were as follows: 

• $379 more was spent for non-Hispanic whites than for American Indians; 

• $293 more was spent for non-Hispanic whites than for blacks; 

• $170 more was spent for non-Hispanic whites than for whites of Hispanic 
origin; 

• at least $300 more was spent for clients between 18 and 24 years old than 
for clients who were age 25 or older, and about $200 more was spent for 
these clients than for clients who were age 17 or younger; 

• more was spent for clients who already had some higher education than 
for clients who had no higher education. 



The findings suggest that after vr agencies provide clients the two 
common initial services, patterns vary widely, since no other service is 
provided to more than a third of the group. In terms of quantity of service, 
at least as measured by the cost of purchased service, states spend more, 
on the average, when clients have severe rather than non-severe 
disabilities, when clients have physical rather than mental disabilities, and 
when clients are white rather than black, Hispanic, or American Indian. 

We are limited in our conclusions by the data at hand. Thus, based on the 
data states are required to submit to rsa, we were unable to determine the 
intensity or cost of any specific category of service that was received. And 
the total cost figure understates the total dollar value of all a client's 
services because the states report neither the cost of counselor time and 
administrative overhead in the vr agency nor the cost of services arranged 
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for by the agency but paid for by other sources. 1 Thus, we could not 
determine whether disparities in the costs of purchased services may be 
made up elsewhere, perhaps through more intensive services provided by 
the agency or services paid for by other sources. And data on services 
alone, no matter how extensive and detailed, do not yield conclusions on 
the key question of the appropriateness of the services to the client's 
needs. 

The data we have presented in this chapter are descriptive and provide no 
direct evidence of discriminatory practices on the part of state vr 
agencies. There are many possible explanations for the differences we 
have uncovered, especially since the groups we have examined differ in 
terms of other characteristics associated with variations in the costs of 
purchased services. 2 Addressing the issue of discrimination is not possible 
with the data at hand and is therefore beyond the scope of this report. 



T?^^™™£^Hati™o We recommend that the broad issue of the adequacy of an average 

KeCOmmenaaUOIlb purchased-service amount of $ 1,573 per client (and of less than $500 for 

half the clients) be an early agenda item for the National Commission on 
Rehabilitation Services authorized in the Rehabilitation Act Amendments 
of 1992 (if it is established). The issue is especially important in view of 
the expansion of eligibility, enacted in the same law, that has the potential 
to stretch the vr budget over an even larger group of clients. 

The limitations in the rsa data and the descriptive evidence of racial 
disparities in purchased service costs prompt us to make two types of 
recommendations to the Rehabilitation Services Administration (rsa). 

First, as part of the data review that we have already (in chapter 

3) recommended be started, as authorized in the 1992 amendments to the 

Rehabilitation Act, rsa should develop plans for improving data on 
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l A study of one year's cases in one state suggests the magnitude of these omitted costs. In 1982, in 
Virginia, the average VR client not only received over $1,600 in purchased services (similar to the 
figure we report), but also over $1,500 in externally funded services and over $300 in counselor 
services. D. Dean and R.C. Dolan, "Using A Better Measure for Services," in M. Berkowitz (ed.), 
Measuring the Efficiency of Public Programs (Philadelphia: Temple University Press, 1988), 186-98. A 
study of all types of services for VR clients in three states later in the 1980's found that the total dollar 
cost was two to three times greater than the cost of purchased services alone. Set? M. Berkowitz et al., 
En hanced Understanding of the Economics of Disability , final report submitted to the National 
Institute of Disability and Rehabilitation Research (Richmond. Va.: Virginia Department of 
Rehabilitative Services. 1988). chapter 5. 

-See appendix 111 for further analysis of differences among clients of different racial groups, using 
other variables measured in the RSA Case Service Reports as well as measures of state economic 
context 
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services received by vr clients. For example, it should be possible, without 
unduly burdening counselors, to routinely collect a more detailed profile 
on each client of the cost, intensity, and frequency of specific services 
provided, purchased, or arranged for by the vr agency. Alternatively, rsa 
could conduct periodic studies of client samples, although the multiple 
disabilities and service types in the vr program would make this approach 
technically complex. 

Second, we recommend that rsa further examine the specific issue of 
racial disparities in spending. This can be done in at least two ways, rsa 
could make use of existing data to shed light on this problem. For 
example, the data from the Client Assistance Program (which exists to 
assist clients who have complaints during the vr process) might be useful 
for determining whether conflicts that exist between clients and the vr 
agency are related to the racial or ethnic group membership of the client. 
Alternatively, evaluations could be designed to include this issue. Thus, 
rsa could ensure that provision is made for the collection of generalizable 
data on the experiences of clients who are members of minority groups in 
the current longitudinal evaluation study, and could focus on the issue in 
future evaluation contracts. Such a focus may well require over-sampling 
members of minority groups, or designing additional data collection 
instruments for measuring processes related to minority issues in 
vocational rehabilitation. 
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The fourth and final evaluation question asked us to describe the results 
achieved through the delivery of vocational rehabilitation (vr) services. 
Traditionally, the vr program has been justified in economic terms as a 
"good investment" Supporters claim that the costs of providing services to 
persons with disabilities are more than balanced by rehabilitated clients' 
improved employment levels and earnings. However, such claims have 
usually been based on analyses of the short-term data collected by the 
state vr agencies. 1 Few studies have examined long-term outcomes; of 
those that did, one looked at clients who received services before the 
implementation of the Rehabilitation Act of 1973, and another at clients 
from only one state agency. 2 Furthermore, little is known about how the 
long-term impact of the program varies for subgroups of clients who differ 
in type and severity of disability, or in other background characteristics. 

We examined the long-term economic outcomes of the program using the 
rsa-ssa "data link" data base containing rsa's information on the 
characteristics of the nearly 865,000 vr applicants whose cases were 
closed in fiscal year 1980, combined with information from ssa on the 
annual earnings of each for the years 1972 through 1988. Thus, the data 
base followed these individuals for several years before referral to the 
program and for 8 years after case closure in 1980. In our analyses, we 
compared the economic outcomes of three groups: (1) clients who were 
rehabilitated, (2) clients who were served but not rehabilitated, and 
(3) clients who were accepted but who dropped out before a service 
program was developed or before services were initiated. We do not claim 
the groups are fully comparable, with no differences other than their vr 
experiences. Yet, with statistical control methods, the second and third 
groups do provide an initial step towards isolating the program's effects as 
well as possible, short of an actual experiment where the study design 
(including randomly assigned experimental and control groups) helps 
eliminate sources of bias that can affect conclusions. Since the data 
included all clients whose cases were closed in the year 1980, we could 



»For example, RSA reported that for the group of clients (without severe disabilities) rehabilitated in 
1988, employment in the competitive labor market jumped 69 percentage points from application to 
closure and average weekly wages rose $163. (For those with severe disabilities, RSA reported a 
63 percentage point gain in the size of the group employed and a $137 per week gain in average 
wages ) Rehabilitation Services Administration, Comparison of Economic Gains Achieved by Persons 
with Severe and Non-severe Disabilities RehabihTatedby State Vocational Rehabilitation Agencies in 
Fiscal Year lfl8& (Washington, D.C.: U.S. Department of Education, 1990). 

Berkeley Planning Associates, Use of the Social Security Data-Link for Assessing the Impact of the 
Federal-State Vocational Rehabilitation Program (Washington, D.C.: U.S. Department of Education, 
January 1$89). Though performed and published in the 198(Vs, this study analyzed a client cohort from 
the mid-1970's, before major changes in the 1973 law were fully implemented. D. Dean and R.C. Dolan, 
"Establishing a Mini-Data Link," in M. Berkowitz (ed.), Measuring the Efficiency of Public Programs 
(Philadelphia: Temple University Press, 1988), 233-55. 
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examine program impact for clients who differed in type and severity of 
disability. (Chapter 1 contains a more detailed description of the data and 
our methods of analysis.) 

Briefly, those applicants who were rehabilitated worked more and earned 
more than their group did before vr, and the trends were better than those 
for non-rehabilitants (partial participants) and dropouts. On the other 
hand, the rise in the proportion of those with earnings in any year was 
short-lived (lasting only 2 years after closure), and subsequent earnings 
gains for the shrinking fraction working were modest. Using statistical 
methods to control for some pre-existing differences between the groups, 
we found the rehabilitants' work level and earnings higher at the 5-year 
point after vr than those for the other two groups. 



Organization Of tllG c ^ a P ter P resents details of our findings in three sections, with results 

^ " in each given separately for clients with physical disabilities, emotional 

OliaptGr disabilities (including mental illness or substance abuse), and mental 

retardation. 3 In the first section, we examine short-term economic 
outcomes, using rsa data alone to show clients' status at the time of 
closure. In the second section, we examine three indicators of long-term 
economic outcomes, covering 8 years after case closure — how many 
worked, how continuously they worked, and what they earned. In this 
section, we also present the results of more detailed statistical analyses of 
the effect of the program, taking account of differences among the groups 
other than their vr experience. In the third section, we assess the 
program's effect on long-term outcomes separately for those with severe 
and non-severe disabilities. 



The state agency records one of three outcomes for clients accepted into 
the vr program. Clients may (1) drop out before a service plan is 
developed or before services are initiated, (2) receive some services but 
not be rehabilitated, or (3) be rehabilitated. Clients are classified as 
rehabilitated if they are engaged in an occupation commensurate with 
their abilities for 60 days after the provision of vr services. 

Approximately 70 percent of clients with physical disabilities (including 
visual, hearing, and orthopedic impairments, amputations, and chronic 



3 In the previous chapter, we found that clients with different types of disabilities received VR services 
that varied in type and cost We also expected that these client groups would face different barriers to 
employment Thus, we analyzed outcomes for these groups separately also; aggregate analysis of all 
clients in the data base would not reveal any differences by disability. 
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illnesses) and clients with mental retardation were rehabilitated in 1980. 
About half the clients *vith emotional disabilities (those due to substance 
abuse or mental illness) were rehabilitated. Roughly 10 percent in each of 
the three disability groups dropped out of the program. The remaining 
clients received some services but were not rehabilitated. The results, 
presented by disability group, are contained in figure 5.1. 



Figure 5.1: Short-Term Outcomes of ■H^^^B 
the VR Program Parcant in group 




Physically Emotionally Mantally 

ditablad disabled raUrdad 

Type of cllenta 



Rehabilitated 




Not ret*' Militated 
Dropped out 



Source: GAO analysis of 1980 combined RSA-SSA data base 



For rehabilitated clients, rsa also records whether they find work in the 
competitive labor market, are placed in sheltered employment situations 
(defined as those settings where employers are allowed to pay less than 
the minimum wage, and where most employees are persons with 
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disabilities), become homemakers (rsa recognizes achieving greater 
independence at home as an acceptable rehabilitation outcome for some 
clients), or are engaged in self-employed or unpaid work. As can be seen in 
table 5.1, a majority of clients who were rehabilitated found work in the 
competitive labor market. However, a substantial percentage of clients 
with mental retardation were placed in sheltered employment. Also, 
across all three disability groups, men were more likely than women to 
find competitive employment. A greater proportion of women than men 
were rehabilitated as homemakers, a pattern that was especially 
pronounced among clients with physical disabilities. 



Table 5.1 : Types of Short-Term Employment Outcomes for Rehabilitated Clients 



Rehabilitants with Rehabilitants with Rehabilitants with 
physical disabilities emotional disabilities mental retardation 



Type of outcome 


Men 


Women 


Men 


Women 


Men 


Women 


Competitive employment 


83% 


67% 


92% 


82% 


74% 


58% 


Homemaker 


8 


29 


2 


13 


1 


11 


Sheltered work 


2 


2 


4 


3 


24 


29 


Other 


7 


3 


3 


2 


2 


2 


Total - 


100 


101 


101 


100 


101 


100 



a Totals may not equal 100 percent due to rounding. 



Source: GAO analysis of 1980 combined RSA-SSA data base 



T nntf-Tprm OlitmmP^ Using the ssa data on vr clients' annual earnings, we examined three basic 
lg leilUVJUltUIl L^t) questions about their experience after the program (with comparisons to 

the years before vr as appropriate): 4 



• How many vr clients worked each year? 

• How continuously did they work across the 8 years? 

• What did they earn? 

Employing these questions, we examined the vr program's effect by 
comparing rehabilitants, non-rehabilitants, and dropouts; we also looked 
for differential effects within the client groups by comparing the three 
major disability groups. 



4 Analysis of clients' pro-program situation would be much stronger if, in addition to basic wage data, 
we also knew the date of onset of disability. This information is not included in RSA data, however. 
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The Proportion Of VR It is noteworthy that, among those rehabilitated in all three disability 

Clients With Any Earnings groups, quite a large fraction had earnings in the year of referral (65 to 
From Wages " 7 ^ P ercent ) 311(1 831 even larger percentage in the year of closure (75 to 

84 percent). However, the increases were temporary; the proportion with 
any earnings dropped to near or below pre-program levels within 2 years 
following closure. These patterns are shown in figure 5.2. 
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Figure 5.2: Percent of Clients With Any 
Earnings From Wagss 



CUsnts With Physical Disabilities 



100 Par cant 
90 




Clients With Emotional Disabilities 




Clients With Mental Retardation 




2 ytan bafora 1 ytar bafort 
referraT refarret* 



Ytar of 
referral" 



Ytar of 
cloture* 



1982 



1985 



1988 



Rehabilitated 



Not rehabilitated 



• • Dropped out 
(Figure notes on next page) 
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a Year of referral varies across clients. 

b Clients' cases were closed in fiscal year 1980, which included parts of calendar years 1979 and 
1980. 

Source: GAO analysis of 1980 combined RSA-SSA data base 



In contrast, clients who were not rehabilitated or who dropped out of the 
program were less likely to have earnings from employment in the years 
after closure from the program than in the years before referral. The 
declines occurred within the first 2 years after closure and then leveled off 
or were slightly reversed. At the end of the period that we studied, from 40 
to 50 percent of those clients who were not rehabilitated or who dropped 
out of the program had some earnings from employment. This compared 
with the 61 to 66 percent of rehabilitated clients who had earnings. 

The post-program patterns of sharp declines, followed by a leveling off, or 
a slight increase, were similar for clients who were not rehabilitated and 
clients who dropped out of the program. However, the proportion 
employed in the former group was slightly lower than in the latter. 



Consecutive Years Of We also examined vr clients' continuity of wage-earning after 

Post-Program Earnings participation. These results are presented in figure 5.3 and are quite similar 

to the results we saw in the previous section. That is, in the years after 
closure, a shrinking fraction of clients showed a record of uninterrupted 
earnings; at the 8-year point, only about a third had earned wages each 
year. 5 Across all three disability types, clients who were rehabilitated were 
much more likely to work continuously (according to this measure) than 
were those in the other two groups (non-rehabilitants and dropouts) — by 
about two to one. 



r The measure of continuity is limited by the available data. The SSA wage information consists of a 
single figure for an entire year. Thus, even those with earnings may not have had steady jobs 
throughout the year. 
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Figure 5.3: Continuity of Earnings 
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(Figure notes on next page) 
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Notes: 

Clients' cases wero closed in fiscal year 1980, which included parts of calendar years 1979 and 
1980. 

Percentages shown for each year represent those with some wages from employment in that year 
and all preceding years since closure. 

Source: GAO analysis of 1980 combined RSA-SSA data base 



Annual Earnings Although the group earning wages shrank noticeably over the years, as 

previously reported, average earnings generally rose. We found this to be 
true for each of the three m^jor disability groups and for rehabilitants, 
non-rehabilitants, and dropouts as well. We first examined earnings for the 
entire group of 1980 closures in the rsa-ssa file — that is, for those of 
working age each year — and noted increases from 1980 to 1988 of $547 to 
$1,430 (depending on the disability type) in the annual average earned by 
rehabilitants. (See figure 5.4.) These averages are lowered by including in 
the calculation for each year many individuals who earned no wages, so 
we recalculated each year's average excluding those with no wages — with 
the results shown in figure 5.5. We found increases of $2,052 to $4,592 
(again depending on the disability type) for rehabilitants over the 8 years. 
Both analyses show that earnings drop-offs before referral to vr were 
reversed following closure. 6 And the figures also show that, while both 
dropouts and non-rehabilitants had wage gains, rehabilitants did much 
better (with gains at least four times as great as gains in other groups), 
with the average eventually exceeding that of the bef ore-referral period. 
(Some of the other closure groups also recouped earlier wage losses.) We 
saw earlier that fewer non-rehabilit **nts and dropouts worked than 
rehabilitants; thus, more had zero earnings. However, even removing these 
individuals from the analysis — as we did for figure 5.5 — shows that 
rehabilitants who were working earned on the average more than those in 
the other groups who worked. 



Evaluations of job training programs have documented how participation is not random, and 
specifically that individuals whose earnings are unusually low just prior to participation in training are 
the ones most likely to enter training. This dip in earnings introduces bias in estimates of program 
effects. Not all VR client groups show a pre-program earnings dip in the SSA annual wage data, but 
some do. (The week-of-application earnings figure used by RSA may be more affected.) We report in a 
later section of this chapter the results of the regression analyses we did to begin to deal with 
pre-program factors; further work with RSA-SSA data could explore additional statistical approaches 
to these problem? common to non-experimental data in order to yield better estimates of the 
program's net effect 
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Figure 5.4: Average Real Annual 
Earnings From Wages 
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(Figure notes on next page) 
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Notes: 

Earnings are adjusted to 1988 dollars, using the consumer price index. 

Each entry is the average for all working-age clients. (Clients with $0 in annual earnings are 
included m the computation.) 

Year of referral varies across clients. 

Clients' cases were closed in fiscal year 1980, which included parts of calendar years 1979 and 



1980. 



Source: GAO analysis of 1980 combined RSA-SSA data base 
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Figure 5.5: Average Real Annua! 
Earnings From Wages for Clients With 
Some Earnings 
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(Figure notes on next page) 
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Notes: 

Earnings are adjusted to 1988 dollars, using the consumer price index. 

Each entry is the average for a(! working-age clients with more than $0 in annual earnings. 

Year of referral varies across clients. 

Clients' cases were closed in fiscal year 1980, which included parts of calendar years 1979 and 
1980. 

Source: GAO analysis of 1980 combined RSA-SSA data base 



Differences in Pre-Program 
Employment for 
Rehabilitate and the 
Comparison Groups 



But the figures portraying these differences also reveal that the groups 
differed before they entered the program. For example, as shown in figure 
5.2: 

• among clients with physical disabilities, declines in employment levels 
among rehabilitants during the 2 years before referral contrasted with 
sharper declines for non-rehabilitants and dropouts; 

• among clients with emotional disabilities, pre-program increases in 
employment levels for rehabilitants contrasted with declines for the other 
groups; and 

• among clients with mental retardation, large increases in pre-program 
employment for rehabilitants contrasted with smaller increases for the 
other groups. 

When groups differ before they take part in a program, it is unclear how to 
interpret post-program differences. Where an experimental design is not 
possible, as in our case, statistical analysis techniques were available that 
have the same goal of controlling for the effect of other factors that could 
affect the results, in order to draw conclusions about a program's 
influence. We used two forms of regression analysis to explore whether 
the vr program had an effect even when considering pre-program 
differences. Such analyses depend on having data on the other factors that 
may be influencing the results. In this case, we could examine only a few 
such factors since we were limited by the data on each client that were 



As we have shown in the three preceding sections, clients considered 
rehabilitated in later years did fare differently from the non-rehabilitants 
and dropouts with whom we compared them: In any one year, more 
rehabilitants worked for wages; more had wages in consecutive years; and 
they earned more. This was true for rehabilitants in all three categories of 
disability. 
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included in the rsa-SSA merged file. 7 Thus, our conclusions about the net 
effect of the vr program are more tentative than if we had additional 
measures of key dimensions related to results, such as a client's 
motivation or family support. 



VR Program Effect on Using regression analysis to examine the three disability groups in 1985, 

Level of Employment we concluded that, even considering all the other factors we could 

measure, an individual who completed a vr program and who was 
considered to be rehabilitated would be significantly more likely than a 
dropout to be working for wages 5 years after closure. Our estimate of the 
program's effect— the estimated difference in likelihood of work— was 
largest for clients with mental retardation, a difference of 19 percentage 
points. We estimated the difference in the likelihood of working for those 
with emotional disabilities at 15 percentage points, and at 12 percentage 
points for those with physical disabilities. 

In our second comparison, however, we found no program effect. That is, 
using a similar analysis involving statistical controls for other factors, the 
non-rehabilitated group (across all three disability types) did not look 
much different from the dropouts at the 5-year point in their likelihood of 
working for wages. Thus, disregarding the diagnosis and counseling that 
both groups received, the purchased services that non-rehabilitants 
received did not make them any more likely than dropouts to be working. 1 



Effect of Program 
Participation on Annual 
Earnings 5 Years After 
Closure 



Again using regression analysis, we examined whether a second vr 
outcome — in this case, the earnings gain for rehabilitants— was 
statistically significant, given the differences between the three closure 
groups in pre-program earnings and in other characteristics. (Details of 
our method and results are contained in appendix IV.) We found that, on 
average, clients with physical disabilities, emotional disabilities, and 
mental retardation were likely to earn about $2,000, $1,600, and $1,000 
more, respectively, at the 5-year point after closure than clients who 
dropped out— a statistically significant difference. 
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7 FuII details of the variables we used and the results of our analyses are in appendix IV. The available 
data on the RSA-SSA file allowed us to examine some demographic variables, as well as pre-program 
work history and earnings. We also included the region of the country where the client received 
service and measures of state economic conditions. 

*RSA officials interpret the similar outcomes for the two groups as possibly reflecting the effects of 
diagnosis and counseling that even dropouts receive. 
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As in the previous section, this second regression showed no vr program 
effect on the wages of clients who were not rehabilitated compared with 
clients who dropped out, once all the other factors were included in the 
analysis. 



Non-Rehabilit^nts 
Comprise a Large 
Proportion of the VR 
Clientele 



The findings on the low level of post-program employment and earnings 
for the non-rehabilitant group are important simply because of the size of 
this group. They comprise from one fifth to one third of the clients in each 
of the three disability groups. (See figure 5.1.) The program invested 
considerable resources to purchase services for this group, and the clients 
in this group invested considerable time in the program. Concerning the 
dollar level of that investment, the state agency spent on the average only 
about one-fifth to one-quarter less on all purchased services for the 
non-rehabilitants than for the rehabilitants. (See table 5.4.) This amount 
was at least four times as much as was spent for the dropouts. And 
non-rehabilitants spent more time as clients of the program (measured as 
time elapsed between program referral and program closure) than did 
either rehabilitants or dropouts. 



Table 5.4: Average Amount Spent on Purchased Services and Average Number of Years Between Program Referral and 
Closure 



Physically disabled 



Emotionally disabled 



Mentally retarded 



Services 



Non- Non- Non- 

Rehab rehab Dropout Rehab rehab Dropout Rehab rehab Dropout 



Average cost of purchased services 



$1,225 



$998 



$133 



$921 



$669 



$133 $1,193 $937 $224 



Average number of years between referral 

and closure 1.93 2.53 



1.42 



1.78 



2.05 



1.20 



2.40 



2.73 



1.58 



Source: GAO analysis of 1980 combined RSA-SSA data base 



Program Effect 
Differs for Severely 
Disabled Clients 



Because the program has a congressional mandate to focus services on 
those clients with more severe disabilities, we examined whether the 
program effect differed for clients with severe and non-severe disabilities. 
We did so by repeating the regressions for employment status and 
earnings in 1985 separately for clients with severe and non-severe 
disabilities (using the same set of variables that we used in the earlier 
analyses). 



We found the program did have statistically significant larger effects for 
those with severe disabilities (when compared with dropouts with severe 
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disabilities)— slightly larger in the case of the likelihood of working at the 
5-year point, and quite a bit larger in the case of average annual wages 
earned at that point. The earnings increase, however, was only 
experienced by those clients with physical or emotional disabilities. The 
effects on earnings for mentally retarded clients were reversed: The gap 
between rehabilitants and dropouts was twice as large for clients with 
non-severe disabilities as for clients with severe disabilities. These findings 
suggest that the program may do well in getting severely mentally retarded 
clients into jobs, but may have greater difficulty in finding them jobs that 
pay much more than those held by program dropouts. 



The analyses in this chapter suggest that the gains in economic status 



Conclusions made by c ^ ents rehabilitated in 1980 were quite temporary. Within the 

group rsa classified as rehabilitated (60 days from the end of services), 
after 2 years the proportion with any earnings from wages returned to near 
or below pre-program levels. The earnings gains for all rehabilitants were 
modest. Although rehabilitated clients may have achieved other valuable 
non-economic outcomes, the long-term economic gains for this group 
were disappointing, particularly when contrasted with the large short-term 
gains shown in previous rsa analyses. 9 

We did find that clients who were rehabilitated after participation in the vb 
program were better off, in economic terms, than clients who did not 
participate. This pattern was consistent across all three major disability 
groups, although the absolute levels of labor force participation and 
earnings that were attained differed. (And post-program differences may 
to some degree reflect pre-program differences that we could not take 
fully into account in our analyses. We also were unable to examine, for 
lack of data, what aspects of participation in the vr program influenced 
post-program employment experience.) 

Although the results suggest that the vr program had a positive effect on 
rehabilitants, that conclusion must be tempered by our findings on 
non-rehabilitants. We found that the large group of clients who 
participated in the program but were ultimately classified as not 
rehabilitated did no better economically than clients who were accepted 
but dropped out of the program. 
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*RSA does not record other non^conomic client outcomes, so we could not include them m our 
evaluation Significant non-economic outcomes include increased job satisfaction, personal 
independence and mobility, the independence of other family members, and increased integration into 
the community. It may become more important to measure these non^economic outcomes 
systematically as the program serves an ever wider and more diverse group. 
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Explaining this finding is beyond the scope of our analysis. On the one 
hand, it is possible that this group of clients did not receive appropriate 
services from the vr program. On the other hand, they may have differed 
systematically from successful participants in terms of such unmeasured 
factors as motivation, and thus could be expected to do less well in spite 
of receiving appropriate services. Evaluating these different explanations 
would require additional data not available in the rsa client file on specific 
services received and on client motivation. 



We make three recommendations based on the findings given in this 
chapter. First, more links of vr files with long-term wage data held by the 
Social Security Administration are needed. We recommend that the 
Secretary of Education negotiate an agreement with the Secretary of 
Health and Human Services to implement the provision of the 
Rehabilitation Act Amendments of 1992 calling for further data links. 
Examining more recent cohorts would show whether the program's 
effects have been maintained as program funds are spread ever thinner, 
and as larger and larger proportions of clients with severe disabilities 
participate in the program. All the rsa Case Service Report data on client 
services should be included in the files used for computer matching; this 
will permit the Secretary of Education to explore in as much detail as 
possible (given the limitations of the data states are now required to send 
rsa) how differences in vr services affect clients. More complex statistical 
analyses of further data links could help control even better for 
pre-program differences. 

Second, we recommend that the Secretary of Education continue the 
longitudinal evaluation of the vr program authorized in the Rehabilitation 
Act Amendments of 1992. The study is an important opportunity to follow 
up on a number of issues raised by our findings. (As we recommend in 
chapter 3, starting the study with a group comprised of individuals just 
becoming interested in rehabilitation will help trace referral and 
acceptance processes.) In addition, effort should then be made to detail 
the experiences of participants and non-participants so that outcomes can 
be linked to the services received from vr or from other providers. Use of 
a comparison group, attention to program process, and sufficient 
information on pre-program experiences and post-program outcomes 
should lead to a greater understanding of why some clients have long-term 
success, as well as help guide policy with regard to the achievement of this 
goal. 
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Finally, we recommend that the Secretary of Education take steps to 
initiate the National Commission on Rehabilitation Services authorized by 
the Rehabilitation Act Amendments of 1992. Our data show modest results 
from the vr program as currently designed and implemented, and changes 
in the 1992 law expanding eligibility are likely to bring further challenges. 
A broadly representative group supported by the structure and resources 
provided by the Commission could weigh the available data, as well as the 
issues of goals, resources, and results that must be considered in a full 
review of the program, before the next reauthorization. Such 
reconsideration is needed not only because of the mixed results in terms 
of program effectiveness, but also because the situation with regard to 
employment of persons with disabilities may be changing rapidly as a 
result of the Americans With Disabilities Act that went into effect in stages 
in 1992-93. The Commission's discussions and recommendations could 
help ensure that the Congress has a well-considered plan, based on a 
thorough review, when the vr program is next reauthorized. 
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Comments From the Department of 
Education and GAO's Response 



Officials of the Rehabilitation Services Administration (rsa) provided oral 
comments on our findings, conclusions, and recommendations. Beyond 
the points noted, they generally found no errors of fact or analysis and did 
not take issue with the recommendations. 

Concerning chapter 2, rsa officials noted that other data sources, such as 
the Current Population Survey (CPs), suggest that prevalence of disability 
is much lower, perhaps 10 to 11 million, in contrast to the 14 to 18 million 
figure from the surveys we cite in chapter 2. We pointed out the warnings 
issued by the Bureau of the Census against using CPS to estimate 
prevalence, and we have added a note in the text explaining why we did 
not use CPS figures. In any case, whatever the population estimate turns 
out to be, it is clear that only a small fraction is now served and that an 
increased proportion of persons with disabilities will be eligible under the 
new law. 

With regard to chapter 4, rsa officials said they were aware of the racial 
and ethnic group disparities in the amounts of purchased sendees that we 
found and were already conducting analyses to understand these 
disparities better. 

Concerning chapter 5, rsa officials stated that some improvement in rsa's 
outcome measures may be expected in the next several years. They noted 
that section 127 of the Rehabilitation Act Amendments of 1992 requires the 
commissioner to establish and publish, by September 30, 1994, evaluation 
standards and performance indicators (including outcomes) for the 
program. They believed that such improvements as following clients for a 
longer period after closure and learning more about their post-program 
jobs, will be considered in developing these standards. 

In addition, rsa officials stated that a comparison in our draft report 
between wages of vr clients and wages in the general population was 
inappropriate. In order to avoid misunderstanding, we removed that 
specific analysis from the final text; however, the issue of 
benchmarks — that is, what are reasonable expectations, in economic 
terms, for rehabilitants — will persist, rsa officials also said they do not 
routinely contrast dropouts and non-rehabilitants, as we do in this report. 
They believe dropouts get substantial non-purchased services (in 
diagnosis, and evaluation and counseling). The non-rehabilitated are, 
however, a large group of clients who spent considerable amounts of time 
in vr and were accorded considerable amounts of purchased services. We 
therefore continue to believe this group should be a focus of study and 
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evaluation so that more may be learned about how resources could be 
used more effectively. 

Concerning the possibilities of additional linkages of rsa and ssa data, the 
rsa officials said there had been progress made. As of June 1993, a 
memorandum of understanding between the Departments of Education 
and Health and Human Services existed in draft form. 

Concerning our several recommendations for analyses to be conducted by 
the yet-to-be-established National Commission on Rehabilitation Services, 
the officials pointed out that it would be hard for rsa to provide much 
support for the commission given the limited resources of the agency. We, 
however, made no change in our recommendation since we believe the 
opportunity the commission provides for sustained review of the vr 
program's services and results is important in light of changing eligibility 
criteria and the implementation of the Americans With Disabilities Act. 
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Type 


Impairment 


Visual 


Blindness, both eyes, no light perception 




Blindness, both eyes (with correction not 




more than 20/200 in better eye or limitation 




in field within 20 degrees) 


Blindness, one eye, other eye defective 


Blindness, one eye, other eye good 


Other visual impairments 


Hearing 


Deafness, prelingual 


Deafness, prevocational 


Deafness, postvocational 


Hard of hearing, prelingual 


Hard of hearing, prevocational 


Hard of hearing, postvocational 


Orthopedic 


Involving three or more limbs or entire body 




Involving one upper and one lower limb 




(including side) 




Involving one or both upper limbs 




(including hands, fingers, and thumbs) 




Involving one or both lower limbs 




(including feet and toes) 




Other and ill-defined impairments 




(including trunk, back, and spine) 




Loss of at least one upper and one lower 




major extremity (including hands, thumbs, 




and feet) 




Loss of both major upper extremities 




(including hands or thumbs) 




Loss of one major upper extremity 




(including hand or thumb) 




Loss of one or both major lower extremities 




(including feet) 




Loss of other and unspecified parts 




(including fingers and toes, but excluding 




thumbs) 


Mental illness 


Psychotic disorders 


Neurotic disorders 




Mental and emotional disorders not 




elsewhere classified 


Mental retardation 


Mental retardation, mild 


Mental retardation, moderate 


Mental retardation, severe 


Substance abuse 


Alcohol abuse or dependence 



(continued) 
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Type 


Impairment 


Other drug abuse or dependence 


Other conditions 


Other conditions resulting from neoplasms 




Allergic, endocrine, metabolic, and 




nutritional diseases 




Diseases of the blood and blood-forming 




organs 




Other specified disorders of the nervous 




system 


Cardiac and circulatory system conditions 


Respiratory system conditions 


Digestive system conditions 


Genito-urinary system conditions 


Speech impairments 


All other disabling diseases and conditions 
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As discussed in chapter 4, we found differences in the amount spent for 
services for people in different racial groups; however, we could not 
explain these differences. One possible explanation is discrimination in 
decisions, but data were not available either to support or rebut such a 
conclusion. A number of other explanations are possible, and this 
appendix shows data pertinent to two, drawn from the rsa 1988 Case 
Service Reports. 

First, we considered that the different racial groups of vr clients may enter 
the vr system with different disabilities or with other background 
differences. Spending differences thus could reflect different needs 
associated with the preexisting differences. Table III.l shows whether 
1988 clients of different races differ on a number of individual 
characteristics, such as disability type and severity, age, and education. 1 
There are some differences, but they are not large. 



Table 111.1: Client Characteristics, by Racial Group 



Race 











American 
Indian/ 


'sian 
American/ 




Client characteristic 


White, non- 
Hispanic 


Hispanic 


Black 


Alaskan 
native 


Pacific 
islander 


All cases 


Closure status 


Rehabilitated 


63% 


65% 


58% 


52% 


65% 


62% 


Not rehabilitated 


28 


30 


35 


38 


28 


30 


Dropped out 


8 


5 


7 


10 


7 


8 


Tvoe of disability 


Visual 


7 


8 


7 


7 


7 


7 


Hearing 


8 


7 


5 


4 


11 


7 


Orthopedic/ amputation 


26 


25 


16 


24 


25 


24 


Mental illness 


19 


17 


20 


15 


23 


19 


Substance abuse 


11 


15 


12 


22 


5 


11 


Mental retardation 


11 


8 


21 


12 


13 


13 


Other disability 


18 


19 


19 


16 


17 


19 


Severely disabled 


65 


62 


65 


62 


65 


65 


Had secondary disability 


42 


42 


40 


48 


38 


42 


(continued) 



l RSA coded the race of the client as (1) white, (2) black, (3) American Indian or Alaskan native, or 
(4) Asian American or Pacific islander. RSA also recorded whether the client was of Hispanic origin. 
We combined these variables to create a new race/ethnicity variable, with the following categories: (1) 
white, not of Hispanic origin; (2) white, of Hispanic origin; (3) black, not of Hispanic origin; 
(4) American Indian or Alaskan Native, not of Hispanic origin; and (5) Asian American or Pacific 
Islander, not of Hispanic origin. 
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Race 



Client characteristic 


White, non- 
Hispanic 


Hispanic 


Black 


American 
Indian/ 
Alaskan 
native 


Asian 
American/ 
Pacific 
islander 


All cases 


Male 


57% 


63% 


57% 


59% 


60% 


58% 


Age at application 


Under 18 


9 


5 


13 


7 


11 


9 


18 to 24 


25 


26 


26 


27 


25 


25 


25 to 34 


29 


31 


30 


28 


28 


29 


35 to 64 


36 


37 


31 


36 


35 


35 


65 and older 


3 


1 


1 


1 


1 


2 


Years of education at application 


Special education 3 


13 


10 


26 


15 


15 


15 


8 or fewer 


8 


18 


9 


10 


12 


9 


9 to 11 


20 


26 


23 


21 


20 


21 


High school graduate 


42 


34 


32 


40 


32 


39 


Some college 


13 


10 


10 


13 


12 


12 


College graduate 


5 


2 


2 


o 


8 


4 


Married 


28 


33 


16 


25 


28 


26 


Live in South 


45 


45 


67 


37 


19 


49 


Received public assistance during VR 
program 13 


30 


30 


33 


35 


40 


31 


Service categories received during VR 














program 














Training 0 


27 


36 


26 


32 


28 


27 


Higher education 


12 


8 


6 


11 


8 


11 


Adjustment training 


18 


20 


24 


18 


17 


19 


Restoration 


34 


32 


31 


31 


22 


33 


No purchased services 


9 


6 


10 


5 


26 


9 



Note: Table entries are percentages with the specific demographic characteristic, in each racial 
group. 



■All clients with a primary or secondary disability of mental retardation were coded as "special 
education" on this variable. 

includes Supplemental Security Income (for reasons of blindness, disability, o. age), Social 
Security Disability Insurance, Veterans' Disability Payments, all other payments for reasons of 
disability (such as Worker's Compensation, or payments from the Black Lung Program), Aid to 
Families with Dependent Children, state general assistance, and all other public supports. 

includes the service categories of (1) business/vocational training. (2) on-the-job training, and 
(3) miscellaneous training. 

Source: GAO analysis of 1988 RSA Case Service Reports 
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For a second possibility, we considered that the differences in spending 
may not arise from individual differences at all, but from conditions where 
the clients live. That is, clients of different races may not be distributed in 
the same way as the general client population among states but may be 
concentrated in richer or poorer states that differ in level of spending on 
vr services (since some of each state's total vr budget comes from state 
sources). We used one direct measure of vr spending and two more 
general indicators of state economic health to explore this coryecture 
about the importance of the state economic context. The results are 
shown in table III.2. 



Table 111.2: State-Level Indicators: Economic Context for VR Clients in Different Racial Groups 








Race 








State level economic indicator 


White, non- 
Hispanic 


Hispanic 


Black 


American 
Indian/ 
Alaskan 
native 


Asian 
American/ 
Pacific 
islander 


All cases 


Per capita income indicator (year of referral) 3 


$13,961 


$15,111 


$13,713 


$13,655 


$14,903 


$13,985 


Unemployment rate 
(year of referral ) b 


7.1% 


6.9% 


7.1% 


7.0% 


6.5% 


7.1 


State VR program spending indicator 


$1,591 


$1 ,567 


$1 ,504 


$1,519 


$1 ,370 


$1,570 


Note: Table entries are variable means, for each racial group. 



a From tables published by the U.S. Department of Commerce, Bureau of Economic Analysis. 
c From tables published by the U.S. Department of Labor, Bureau of Labor Statistics. 
Source: GAO analysis of 1988 RSA Case Service Reports 



We first asked what the typical state vr spending level, and the typical 
state income and unemployment levels, were for the full set of clients. We 
did this by assigning to each of the 344,865 accepted clients in the 1988 
Case Service file his or her own state's figure— that is, we assigned vr 
client A, living in Texas, the Texas statewide per capita income figure for 
all citizens, the states ride unemployment figure, and the state average vr 
spending on purchased services for all clients: we assigned client B, from 
Maine, that state's figures; and so on. Then, we calculated an overall 
average for each of the three indicators across all clients. Those results are 
shown in the right-hand column of table III.2, "all cases." 

To address the main objective of this analysis, we next checked each 
racial subgroup on our three indicators. For example, when we separated 
Hispanic clients and, as before, developed an average state income 
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indicator just for that group (again using for each individual his or her 
state's income figure), the average jumped $1,126 above that for all cases, 
to $15,111. This observation implies nothing about individual Hispanic 
clients' economic well-being. It only means that vr's Hispanic clients in 
1988 were to be found in states generally characterized by higher income, 
which may be understandable given that many Hispanics live in states 
such as New York and California that are among the wealthiest. On the 
other hand, black clients in 1988 — as shown by their lower average state 
income figure — appeared to live in states slightly poorer than the average 
for all clients. 
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Regression Analyses for Long-Term 
Outcomes 



This appendix contains more detailed discussion of the regression 
analyses conducted to examine the impact of vr program participation on 
two dependent variables: (1) the probability of employment in 1985 and 
(2) the level of earnings attained by those clients employed thai year. 
Separate regressions were conducted for clients with physical disabilities, 
emotional disabilities, and mental retardation. We present the variables 
used in both, and then results for the two different, analyses. Finally, we 
show the results of repeating the analyses after separating those with 
severe and non-severe disabilities. 



Variables Used in the 
Regression Analyses 



The regressions included variables from the combined rsa-ssa data base. 
Some variables came from the rsa portion of the file, drawn from the 1980 
Case Service Reports. Others came from the ssa portion of the file, drawn 
from the Summary Earnings Record. In addition, we added to the file two 
variables measuring state-level economic conditions. 



RSA Variables 

Participation in the VR Program 



Race 



Gender 
Age 

Education 



ERLC 



If clients were accepted for vr services, rsa used one of three codes to 
categorize their status when their cases were closed: (1) rehabilitated; 

(2) not rehabilitated, after at least some services were delivered; or 

(3) dropped out before services were delivered. We created two dummy 
variables from these three categories, which we used to measure program 
impact: (1) clients who were rehabilitated versus clients who dropped out, 
and (2) clients who were not rehabilitated versus clients who dropped out. 

rsa coded the race of the client as (1) white, (2) black, (3) American Indian 
or Alaskan native, or (4) Asian American or Pacific islander. We 
constructed three dummy variables (black, American Indian, and Asian 
American) using whites as the omitted category. 

Males were coded as 1, females as 0. 

Age (in years) at time of case closure was included in the rsa portion of 
the file. 

For clients with physical or emotional disabilities, the regressions 
included a variable measuring years of education. The variable was 
omitted in regressions for clients with mental retardation, since years of 
education were not recorded for these clients. In addition, the variable 
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was missing for those clients with primary disabilities that were physical 
or emotional and a secondary disability of mental retardation. 

Severity of Disability We used rsa's categorization of clients as severely or not severely 

disabled. We alsc included a variable that measured whether a client had a 
secondary disabling condition that substantially limited his or her 
employment potential. 

Received Public Assistance A dummy variable measured whether a client received Supplemental 

Security Income (for reasons of blindness, disability, or age), Aid to 
Families with Dependent Children, or state general assistance funds while 
in the vr program. 

Region of the Country We coded each client's state according to which region it is in. (There are 

10 rsa regional offices.) We included nine dummy variables in the 
regressions, using region IX (which includes California, Nevada, Hawaii, 
and the Pacific Islands) as the omitted category. 



State Economic Variables As previously noted, rsa recorded the state in which a client received vr 

services. We added two variables to the file, which measured state 
economic conditions the year clients left the vr program: (1) average per 
capita income (from tables published by the Bureau of Economic Analysis, 
U.S. Department of Commerce) and (2) unemployment rate (from tables 
published by the Bureau of Labor Statistics, U.S. Department of Labor). 
Since these were state-level aggregate variables, the values were the same 
for all clients from the same state. 



SSA Variables 

In the logistic regressions, we included two dummy variables measuring 
whether the client was employed in the year of referral to the program and 
the year prior to the year of referral. These variables were constructed by 
recoding the earnings variables that ssa derived from the Summary 
Earnings Record and included in the file. 

Pre-Program Earnings In the ordinary least squares (ols) regressions, we included two variables 

measuring (1) earnings in the year of referral to the program, and 
(2) earnings in the year prior to the year of referral. Both variables were 
derived from the ssa Summary Earnings Record, and both were acUusted 
to 1988 dollars, using the consumer price index. 



Pre-Program Employment 
Status 



ERLC 
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We used logistic regression to examine whether the observed differences 
between the groups in post-program employment were statistically 
significant, given the preexisting differences between groups. This is the 
appropriate statistical procedure for examining dichotomous dependent 
variables. 

We examined employment levels in 1985, 5 years after clients had 
participated in the vr program. We conducted a separate logistic 
regression for each mgjor disability group (those with physical disabilities, 
emotional disabilties, and mental retardation). We included in each 
regression those variables that were available in the data set and that were 
likely to be related to pre-program differences between the groups, 
including demographic characteristics, pre-program earnings history, 
region of the country where clients received services, and state economic 
conditions. Each regression then tested for program effect by including 
two variables that measured membership in one of the three closure 
categories (rehabilitants, non-rehabilitants, and dropouts). The regression 
coefficients for these variables relate to the differences between the 
closure categories after accounting for the impact of other variables 
included in the regression. We were then able to examine statistically 
whether rehabilitants did better than clients who did not persevere in the 
program (dropouts), and whether clients who persevered but did not 
successfully complete the program (non-rehabilitants') did better than 
clients who did not persevere (dropouts). 

A summary of the results of these analyses is presented in table IV. L (Full 
results of the regressions for each disability group are included in tables 
IV.2, IV.3 and IV.4. Summary statistics for the regressions are in table IV.5.) 
In the first table, we present the logistic regression coefficients for the 
group membership variables, the standard errors associated with these 
coefficients, and their probability levels (whether the coefficient is 
statistically significant). We also present in the table the percent of each 
group that was actually employed in 1985, compared with the percent that 
was predicted based on the logistic regression equations. 
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Table IV.1: Predicting Whether Clientt 
Have Any Earnings From Employment 



Physically disabled 




Rehab 


Non-rehab 


Dropout 


Percent in group 


70 


21 


9 


Percent with any earnings in 1985 


62 


41 


47 


Predicted percent with any 
earnings in 1985 


59 


43 


47 


Differences in predicted 
percentages 


Rehab vs. dropout 




+ 12 




Non-rehab vs. dropout 




-4 




Logistic regression parameter 
estimates 


Coefficient 


Standard error 


Probability 


Rehab vs. dropout 


.58 


.07 


<.001 


Non-rehab vs. dropout 


-.18 


.08 


<.05 
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Emotionally disabled 






Mentally retarded 




Rehab 


Non-rehab 


Dropout 


Rehab 


Non-rehab 


Dropout 


55 


35 


10 


67 


25 


8 


64 


45 


49 


66 


44 


46 


62 


46 


47 


61 


39 


42 






+15 






+ 19 






-1 






-3 




Coefficient 


Standard error 


Probability 


Coefficient 


Standard error 


Probability 


.67 


.06 


<.001 


.81 


.07 


<.001 


-.06 


.06 


a 


-.14 


.08 


a 



•Regression coefficient not significant at p < .05. 

Source: GAO analysis of 1980 combined RSA-SSA data base 
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Table IV.2: Logistic Regression Analysis Predicting Whether Physically Disabled Clients Were Employed In 1985 


Variable 


Regression 
coefficient 


Coefficient 
standard error 


Wald statistic Probability level 


Participation in VR program (vs. dropouts) 


Rehabilitants 


.5798 


.0694 


69.72 


<.001 


Non-rehabi!itants 


-.1802 


.0776 


5.40 


<.05 


Race (vs white) 


Black 


-.1132 


.0509 


4.95 


<.05 


American Indian 


.0619 


.2531 


0.06 


a 


Asian 


-.0414 


.1739 


0.06 


a 


Gender (male) 


.1391 


.0405 


11.78 


<.001 


Age at closure 


-.0425 


.0019 


500.77 


<.001 


Education (in years) 


.0633 


.0081 


61.75 


<.001 


Severely disabled 


-.5830 


.0404 


208.72 


<.001 


Had secondary disability 


-.1549 


.0431 


12.89 


<.001 


On public assistance 


-.2728 


.0501 


29.63 


<.001 


State per capita income, year of closure 


.0000066 


.0000178 


0.14 


a 


State unemployment rate, year of closure 


-.0245 


.0145 


2.84 


a 


Region (vs. region IX — California, 
Nevada, Hawaii) 


I. New England 


.1552 


.1469 


1.12 


a 


II. New York and New Jersey 


.3632 


.1032 


12.39 


<.001 


III. Border states 


-.0562 


.0915 


0.38 


a 


IV. Old South 


.0729 


.1146 


0.40 


a 


V. Great Lakes 


.0782 


.0899 


0.76 


a 


VI. Southwest 


.0997 


.1068 


0.87 


a 


VII. Midwest farmbelt 


-.0990 


.1259 


0.62 


a 


VIII. Rocky Mountains 


.0855 


.1409 


0.37 


a 


X. Pacific Northwest 


.2374 


.1117 


4.51 


<.05 


Had job the year of referral 


.6782 


.0465 


212.81 


<.001 


Hod job one year prior to referral 


.5711 


.0487 


137.58 


<.001 




Intercept 


.1955 


.3690 


0.28 


a 



a Wald statistic not significant at p < .05. 



Source: GAO analysis of 1980 combined RSA-SSA data base 
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Table 1V.3: Logistic Regression Analysis Predicting Whether Emotionally Disabled Clients Were Employed In 1985 



Variable 


Regression 
coefficient 


Coefficient 
standard error 


Waid statistic 


Probability level 


Participation in VR program (vs. dropouts) 


Rehabilitants 


.6692 


.0598 


125.34 


<.001 


Non-rehabilitants 


-.0564 


.0619 


0.83 


a 


Race (vs. white) 


Black 


-.1974 


.0434 


20.69 


<.001 


American Indian 


-.0108 


.1810 


0.00 


a 


Asian 


-.0777 


,1605 


0.23 


a 


Gender (male) 


-.1491 


,0350 


18.15 


<.001 


Age at closure 


-.0350 


.0018 


367.83 


<.001 


Education (in years) 


.0501 


,0076 


43,84 


<.001 


Severely disabled 


-.5175 


,0353 


214.93 


<,001 


Had secondary disability 


.0873 


.0369 


5.60 


<.05 


On public assistance 


-.1541 


,0407 


14.32 


<.001 


State per capita income, year of closure 


-.000012 


.0000165 


0.57 


a 


State unemployment rate, year of closure 


-.0274 


,0134 


4.16 


<.05 


Region (vs. region IX— California, 
Nevada, Hawaii) 


L New England 


.4132 


.1172 


12.44 


<.001 


II. New York and New Jersey 


.3612 


,0799 


20,45 


<.001 


III. Border states 


.1675 


.0781 


4,60 


<,05 


IV. Old South 


,1471 


.1026 


2,01 


a 


V. Great Lakes 


.1460 


.0757 


3,72 


a 


VI. Southwest 


.0669 


.0891 


0,56 


a 


VII. Midwest farmbelt 


.1220 


,1051 


1.35 


a 


VIII. Rocky Mountains 


.0608 


.1083 


0.31 


a 


X. Pacific Northwest 


.2799 


,0989 


8.02 


<.01 


Had job the year of referral 


.5156 


.0402 


164.14 


<.001 


Had job one year prior to referral 


.5650 


,0394 


205.79 


<.001 




Intercept 


.3311 


.3361 


0.97 


a 



a Wald statistic not significant at p < 05. 



Source: GAO analysis of 1980 combined RSA-SSA data base 



S4 



ERIC 



Page 83 



GAO/PEMD-93-10 'ederal Vocational Rehabilitation Program Effectiveness 



Appendix IV 

Regression Analyses for Long-Term 
Outcomes 





Table IV.4: Logistic Regression Analysis Predicting Whether Mentally Retarded Clients Were Employed in 1985 


Variable 


Regression 
coefficient 


Coefficient 
standard error 


Wald statistic 


Probability level 


Participation in VR program (vs. dropouts) 


Rehabilitants 


.8098 


.0731 


122.66 


<.001 


Non-rehabilitants 


-.1407 


.0801 


3.08 


a 


Race (vs. white) 


Black 


-.2228 


.0496 


20.16 


<.001 


American Ind'an 


-.3557 


.3190 


1.24 


a 


Asian 


-.1190 


.1834 


0.42 


a 


Gender (male) 


.2326 


.0413 


31.71 


<.001 


Age at closure 


-.0106 


.0023 


20.44 


<.001 


Severely disabled 


-.1810 


.0507 


12.73 


<001 


Had secondary disability 


-.2570 


.0426 


36.33 


<.001 


On public assistance 


-.2054 


.0428 


23.00 


<.001 


State per capita income, year of closure 


-.000023 


.0000208 


1.27 


a 


State unemployment rate, year of closure 


-.0355 


.0151 


5.53 


<.05 


Region (vs. region IX— California, 
Neva^af Hawaii) 


I. New England 


.2262 


.1521 


2.21 


a 


II. New York and New Jersey 


.2819 


.0930 


9.19 


<.01 


III. Border states 


.1417 


.1010 


1.97 


d 


IV. Old South 


.0584 


.1356 


0.19 


a 


V. Great Lakes 


.1998 


.0954 


4.38 


<.05 


VI. Southwest 


-.0418 


.1218 


0.12 


a 


VII Midwest farmbelt 


.1179 


.1253 


0.89 


a 


VIII. Rocky Mountains 


.3880 


.1504 


6.65 


<.01 


X. Pacific 
Northwest 


.0216 


.1258 


0.03 


a 


Had job the year of referral 


.4437 


.0466 


90.64 


<.001 


Had job one year prior to referral 


.2546 


.0456 


31.16 


<.001 


Intercept 


-.4174 


.4058 


1.06 


a 



a Wald statistic not significant at p < 05 



Source GAO analysis of 1980 combined RSA-SSA data base 
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Table 1V.5: Summary Statistics for 
Logistic Regression Analyses 



Statistic 


Physically 
disabled 


Emotionally 
disabled 


Mentally 
retarded 


Percent correctly classified 


70 


66 


64 


-2 x log likelihood of model (degrees of 
freedom) 


15.510.7 3 
(13.227) 


20.115.6 3 
(16.225) 


13,878.2 a 
(10.796) 


Goodness of fit 
(degrees of freedom) 


13.301. 4 a 
(13,227) 


16,297.2 s 
(16.225) 


10.823.3 3 
(10,796) 


Change in chi-square, adding VR 
participation variables 
(degrees of freedom) 


268.1 3 
(2) 


428.7 a 
(2) 


451. 0 a 
(2) 


Number of cases 


13,252 


16.250 


10,820 



3 p< .001. 

Source: GAO analys.s of 1980 combined RSA-SSA data base 



Table IV. 1 shows that clients in all three disability groups who were 
rehabilitated were significantly more likely to have earnings from wages 5 
years after program closure than were clients who dropped out of the 
program. This was indicated by the large positive coefficients and small 
standard errors for the variable "rehabilitants versus dropouts." 

The significant effect of the program is also indicated by the differences in 
the predicted values for rehabilitants and dropouts in each disability 
group. For example, among clients with mental retardation, rehabilitants 
were 19 percentage points more likely to be employed in 1985 than were 
dropouts. This difference remained after explaining as much of the 
preexisting differences between the groups as was possible with the data 
that were available. The effect was smaller for the emotionally disabled 
and physically disabled clients, as can be seen in the table. 

Conversely, for two of the three disability groups, clients who were not 
rehabilitated did not differ from clients who dropped out in the likelihood 
of employment in 1985. Although they participated in the VR program, at 5 
years after program closure, those clients with emotional disabilities and 
with mental retardation were no different from clients who did not 
participate. This finding is indicated by the small coefficients for the 
variable "non-rehabilitants versus dropouts." The predicted probabilities 
of employment in 1985 for dropouts were only slightly higher than for 
non-rehabilitants in the case of clients with mental retardation (3 percent 
difference) and for those with emotional disabilities (1 percent 
difference). 
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non-rehabilitants in the case of clients with mental retardation (3 percent 
difference) and for those with emotional disabilities (1 percent 
difference). 

Clients with physical disabilities who were not rehabilitated, however, 
were less likely to be employed in 1985 than were clients who dropped 
out. For these clients, participation in the program was associated with 
worse outcomes in 1985 than if they had not participated at all. The effect 
was small (the predicted probability of employment was only 4 percent 
less than for dropouts), but nevertheless it was a statistically significant 
difference. 



The approach to this second analysis was similar to that reported in the 
previous section. We included in each ordinary least squares (ols) 
regression those variables available in the data set and likely to be related 
to pre-program differences between the groups, and tested for program 
impact by including two variables that measured membership in one of the 
three closure categories. As before, we were able to examine statistically 
whether rehabilitants earned more than clients who did not persevere in 
the program (dropouts), and the extent to which clients who persevered 
but did not successfully complete the program (non-rehabilitants) differed 
from clients who did not persevere (dropouts). 

Summary results of these analyses are presented in table IV.6. (The full 
results for each disability group are in tables IV.7 through IV.9, with 
summary statistics in table IV. 10.) In table IV.6, we present the ols 
regression coefficients for the group membership variables, the standard 
errors associated with these coefficients, and their probability levels 
(whether the coefficient is statistically significant). Also included are the 
average earnings levels in 1985, which are compared with the predicted 
earnings levels derived from the regression equations. 
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Table IV.6: Predicting Earnings Levels 
for Clients With Any Earnings From 
Employment in 1985 





Physically disabled 






Rehab 


Non-rehab 


Dropout 


Percent in group 


70 


21 


9 


Actual earnings in 1985 


$12,837 


$10,284 


$11,056 


Predicted earnings in 1985 


12,659 


11,065 


10,626 


Parameter estimates (in $) 


Coefficient 


Standard error 


Probability 


Rehab vs. dropout 


+2,033 


409 


<.001 


Non-rehab vs. dropout 


+439 


474 


a 
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Emotionally disabled 






Mentally retarded 




Rehab 


Non-rehab 


Dropout 


Rehab 


Non-rehab 


Dropout 


55 


35 


10 


67 


25 


8 


$10,200 


$7,205 


$8,228 


$6,227 


$4,700 


$5,471 


10,058 


7,977 


8,483 


5,494 


3,990 


4,457 


Coefficient 


Standard error 


Probability 


Coefficient 


Standard error 


Probability 


+ 1,575 


313 


<.001 


+1.037 


290 


<.001 


-506 


336 


a 


-467 


322 


a 



Note: Earnings are adjusted to 1988 dollars, using the consumer price index. 



■Regression coefficient not significant at p < .05. 

Source: GAO analysis of 1980 combined RSA-SSA data base 
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Table IV.7: Ordinary Least Squares Regression Analysis Predicting 1985 Real Annual Earnings of Physically Disabled 
Clients 


Variable 


Regression 
coefficient 


Coefficient 
standard error 


T-statistic Probability level 


Participation in VR program (vs. dropouts) 


Rehabilitants 


2,033.37 


409.35 


4.97 


<,001 


Non-rehabilitants 


438.74 


473.88 


0.93 


a 


Race (vs. white) 


Black 


-1,035.32 


290.52 


-3.56 


<.001 


American Indian 


-3,171.36 


1,420.76 


-2.23 


<.05 


Asian 


-516.58 


960.95 


-0.54 




Gender (male) 


2,355.66 


222.75 


10.58 


<.001 


Age at closure 


-54.36 


10.60 


-5.13 


<.001 


Education (in years) 


686,16 


47.25 


14,52 


<.001 


Severely disabled 


98.51 


217.32 


0.45 


a 


Had secondary disability 


-1,222.36 


243.72 


-5.02 


<.001 


On public assistance 


-733.41 


304.67 


-2,41 


<.05 


State per capita income, year of closure 


0.3964 


0,0957 


4,14 


<.001 


State unemployment rate, year of closure 


-11.07 


78.91 


y ■ 0.14 


a 


Region (vs. region IX— California, Nevada, Hawaii) 


I. New England 


1 ,306.38 


792.20 


1.65 


a 


II. New York and 
New Jersey 


1 ,078,24 


539.11 


2.00 


<.05 


111. Border states 


-351.78 


506,78 


-0.69 


a 


IV. Old South 


-361.16 


625,12 


-0.58 


a 


V. Great Lakes 


292.51 


494.39 


0.59 


a 


VI. Southwest 


628.10 


572.02 


1.10 


a 


VII. Midwest farmbelt 


-944.45 


666.53 


-1.42 


a 


VIII. Rocky 
Mountains 


-631.02 


743.43 


-0.85 


a 


X. Pacific 
Northwest 


-1,243.59 


620.08 


-2.01 


<.05 


Earnings in the year of referral 


0.2036 


0.0204 


9.98 


<.001 


Earnings one year prior to referral 


0.2969 


0.0174 


17.09 


<.001 




Intercept 


-5,161.23 


1,990.05 


-2.59 


<.01 



a T-value not statistically significant at p < .05. 



Source GAO analysis of 1980 combined RSA-SSA data base 
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Table IV.8: Ordinary Least Squares Regression Analysis Predicting 1985 Real Annual Earnings of Emotionally Disabled 




Clients 










Variable 


Regression 
coefficient 


Coefficient 
standard error 


T-statistic Probability level 


Participation in VR program (vs. dropouts) 


Rehabiiitants 


1,574.90 


313.03 


5.03 


<.001 


Non-rehabilitants 


-505.62 


335.76 


-1.51 


a 


Race (vs. white) 


Black 


-438.71 


227.41 


-1.93 


a 


American Indian 


-2,102.92 


925.71 


-2.27 


<.05 


Asian 


359.10 


809.78 


0.44 


a 


Gender (male) 


292.20 


175.92 


1.66 


a 


Age at closure 


-14.05 


9.57 


-1.47 


a 


Education (in years) 


507.25 


40.11 


12.65 


<.001 


Severely disabled 


-1,335.23 


178.45 


-7.48 


<.001 


Had secondary disability 


-47.82 


182.89 


-0.26 


a 


On public assistance 


297.31 


219.58 


1.35 


a 


State per capita income, year of closure 


0.2964 


0.0807 


3.67 


<.001 


State unemployment rate, year of closure 


30.26 


67.01 


0.45 


a 


Region (vs. region IX— California, Nevada, Hawaii) 


I. New England 


573.68 


552.00 


1.04 


a 


ll. New York and 
New Jersey 


607. 94 


416.69 


1.46 


a 


III. Border states 


-868.63 


410.69 


-2.12 


<.05 


IV. Old South 


-664.53 


513.23 


-1.30 


a 


V. Great Lakes 


-909.92 


392.99 


-2.32 


<.05 


VI. Southwest 


-670.72 


445.02 


-1.51 


a 


VII. Midwest 
farmbelt 


-2,103.57 


519.96 


-4.05 


<.001 


VIII Rocky 
Mountains 


-926.27 


544.16 


-1.70 


a 


X. Pacific 
Northwest 


-1,067.69 


509.78 


-2.09 


<.05 


Earnings in the year of referral 


0.3593 


0.0205 


17.53 


<.001 


Earnings one year prior to referral 


0.2309 


0.0173 


13.34 


<.001 




Intercept 


-1,779.74 


1,633.27 


-1.09 


8 



a T-value not statistically significant at p < .05 



Source: GAO analysis of 1980 combined RSA-SSA data base 
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Table IV.9: Ordinary Least Squares Regression Analysis Predicting 1985 Real Annual Earnings of Mentally Retarded 
Clients 


Variable 


Regression 
coefficient 


Coefficient 
standard error 


T-statistic Probability level 


Participation in VR program (vs. dropouts) 


Rehabilitants 


1,037.47 


289.57 


3.58 


<.001 


Non-rehabilitants 


-466.64 


321.59 


-1.45 


a 


Race (vs. white) 


Black 


446.96 


170.67 


2.62 


<.01 


American Indian 


-1,278.31 


1,117.88 


-1.14 


s 


Asian 


-40.06 


684.20 


-0.06 


a 


Gender (male) 


852.99 


148.45 


5.75 


<.001 


Age at closure 


-105.80 


9.93 


-10.66 


<.001 


Severely disabled 


-1,111.95 


154.42 


-7.20 


<.001 


Had secondary disability 


-243.06 


163.38 


-1.49 


a 


On public assistance 


-514.34 


162.63 


-3.16 


<.01 


State per capita income, year of closure 


-0.0475 


0.0672 


-0.71 


a 


State unemployment rate, year of closure 


-63.64 


53.59 


-1.19 


a 


Region (vs. region IX— California, Nevada, Hawaii) 


I. New England 


-237.10 


583.04 


-0.41 


a 


II. New York and New 
Jersey 


-30.95 


366.91 


-0.08 


a 


Ml. Border states 


-1,047.15 


380.26 


-2.75 


<.01 


IV. Old South 


-156.27 


456.05 


-0.34 


a 


V. Great Lakes 


-718.23 


353.50 


-2.03 


<.05 


VI Southwest 


-905.69 


429.46 


-2.11 


<.05 


VII. Midwest farmbelt 


-1,361.82 


436.95 


-3.12 


<.01 


VIII. Rocky Mountains 


-1,787.89 


498.50 


-3.59 


<.001 


X. Pacific Northwest 


-1,465.95 


499.81 


-2.93 


<.01 


Earnings the year of referral 


0.4970 


0 0272 


18.24 


<.001 


Earnings one year prior to referral 


0.2598 


0.0251 


10.36 


<.001 




Intercept 


9,030 55 


1,351.01 


6.68 


<001 



a T-value not statistically significant at p < 05. 



Source. GAO analysis of 1980 combined RSA-SSA data basn 
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Regression Analyses for Long-Term 
Outcomes 



Table IV.10: Summary Statistics for 
Ordinary Least Squares Regression 
Analyses 



Statistic 


Physically 
disabled 


Emotionally 
disabled 


Mentally 
retarded 


R-square 


0.23 


0.20 


0.31 


Adjusted R-square 


0.23 


0.20 


0.31 


Change in R-square, adding VR 
participation variables 


0.005 


0.01 


0.01 


Standard error of estimate 


8,874.51 


8,030.89 


5,320.29 


F-statistic 
(degrees of freedom) 


91.90 a 
(24,7266) 


92.67 a 
(24,8991) 


115.64 a 
(23,5799) 


Number of cases 


7,291 


9,016 


5,823 



a p < .001. 

Source: GAO analysis of 1980 combined RSA-SSA data base 



Rehabilitated clients in all three disability groups had significantly higher 
earnings from wages in 1985 than clients who dropped out of the program, 
when other factors were taken into account (and considering only wage 
earners). This was indicated by the large positive coefficients and small 
standard errors for the variable "rehabilitants versus dropouts." The 
coefficients from the regression analyses are interpreted as follows: 
Clients with physical disabilities, emotional disabilities, and mental 
retardation earned about $2,000, $1,600, and $1,000 more, respectively, 
than did clients who dropped out of the program. Clients who were not 
rehabilitated did not, however, differ from clients who dropped out when 
we examined average earnings in 1985. This finding is indicated by the 
statistically insignificant coefficients for the variable u non-rehabilitants 
versus dropouts." 



Analysis by Severity 
of Disability 



We repeated the regression analyses separating those with severe 
disabilities, with the results shoun in table T7.ll. The long-term impact of 
rehabilitation on whether clients had earnings in 1985 was slightly greater 
for persons with severe disabilities than for persons with non-severe 
disabilities. This can be seen by comparing the size of the logistic 
regression coefficients in horizontal rows 1 and 2 of the table for all three 
disability groups. On the other hand, one can recall from table 5.2 that 
clients with physical disabilities who were not rehabilitated were less 
likely to have earnings from wages in 1985 than were dropouts. As it turns 
out, this effect is a function of the lesser likelihood of employment of 
severely disabled non-rehabilitants with physical disabilities, as can be 
seen in row 3 of table IV. 1 1 . 
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Table IV.11: Program Effects in 1985, 
by Severity of Disability 





Physically disabled 






Coefficient 


Standard error 


Probability 


Have earnings in 1985 


Rehab vs. dropout 


Severe 


+0.64 


0.09 


<.001 


Non-severe 


+0.52 


0.10 


<.001 


Non-rehab vs. 
dropout 


Severe 


-0.28 


0.11 


<.01 


Non-severe 


-0.03 


0.12 


8 


Earnings in 1985 


Rehab vs. dropout 


Severe 


+$2,799 


$631 


<.001 


Non-severe 


+$1,459 


$540 


<.01 


Non-rehab vs. 
dropout 


Severe 


+$567 


$725 


8 


Non-severe 


+$355 


$629 


8 
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Emotionally disabled 






Mentally retarded 




Coefficient 


Standard error 


Probability 


Coeffcient 


Standard error 


Probability 






+0.77 


0.08 


<.001 


+0.93 


0.09 


<.001 


0.54 


0.09 


<.001 


+0.55 


0.14 


<.001 




-0.10 


0.08 


a 


-0.07 


0.10 


a 


-0.03 


0.09 


a 


-0.25 


0.15 


a 






+$1,862 


$446 


<.001 


+$744 


$319 


<.05 


+$1,371 


$436 


<.01 


+$1,449 


$599 


<.05 




-$348 


$476 


a 


-$606 


$352 




-$569 


$469 


a 


-$455 


$674 





degression coefficient not significant at p < ,05. 



Source: GAO analysis of 1980 combined RSA-SSA data base 

Examining earnings, we did similar ols regressions as before and found 
that the long-term impact of rehabilitation on earnings levels in 1985 was 
quite a bit greater for persons with severe disabilities — for two of the three 
disability groups. We found the reverse pattern in the group of clients with 
mental retardation, where earnings at the 5-year point were greater among 
those with non-severe disabilities. These results are discussed further in 
chapter 5. 
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Robert Lerman 

The American University 
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Pelavin Associates 
Washington, D.C. 
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Washington, D.C. 
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Glossary 



Adapted from the transmittal of instructions for the rsa-911 Reporting 
System sent from rsa to state agencies. 



Diagnosis and Evaluation This complex of services is designed to enable the rehabilitation agency to 

determine a client's eligibility for vocational rehabilitation services, and/or 
to determine the nature and scope of services to be provided. Diagnosis 
and evaluation can be medical, psychological, social, or vocational in 
scope. 



Counseling and Guidance This would include any of the many different kinds of counseling and 

guidance services that counselors may have to provide for their clients. 



Restoration This category includes those medical and medically-related services that 

are necessary to correct or substantially modify a physical or mental 
condition. Restoration services include surgery, therapy, treatment, 
hospitalization, prosthetic appliances, etc. 



Adjustment Training This is training that helps the client to adjust to a particular situation 

hindering his or her ability to work. Included would be work conditioning; 
developing work tolerance, training in the use of artificial limbs, aids, or 
appliances; mobility training; remedial training; literacy training; lip 
reading; braille; etc. 

Vocational Training This is noncollegiate postsecondary education. Included is training in a 

business/commercial school or college (preparing the client for work in 
areas of office practice, typing, bookkeeping, accounting, etc.) and a 
vocational trade school (preparing the client for occupations such as 
welding, woodworking, TV repair, drafting, cosmetology, etc.). 



On-The-Job Training Training by a prospective employer in which the client usually works for 

wages while learning the skills of a job. 



College Training All academic training on a level higher than a secondary education. 
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Miscellaneous Training 



Training that does not readily fit into other categories of training (for 
example, academic training on a secondary level or lower, or specialized 
schools for the blind and deaf which are academic in nature). 



Referral 



A job referral occurs when a client is sent for and has a job interview with 
a prospective employer. This referral need not result in the offer of a job. 



Placement 



A placement service is rendered when the client is referred to, and 
subsequently hired by, an employer. Excluded would be instances in 
which the client found his or her own job, or where the client's employer 
at the time of application for rehabilitation services retained the individual 
in employment. A key element of this service is that the client became 
employed as a result of the job referral. 



Transportation 



This is any service that enables the client to arrive at appointments for 
diagnosis and evaluation, medical services, training, or any other 
rehabilitation service, as well as any that permits the client to get to work. 
Included would be the provision of vans, cabs, or private cars for the 
client, as well as payments made to these carriers. 



Income Maintenance 



Included is any service provided to cover basic living expenses so that the 
individual can derive the full benefit of other vocational rehabilitation 
services. 



Other Services 



Included are reader and interpreter services, occupational tools and 
equipment, initial stocks and licenses, services to family members for the 
benefit of the client, and medical care for acute conditions arising during 
rehabilitation that jeopardize a client's rehabilitation potential. 
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